Software Engineer - Big Data (Compute/Data Distiller), Adobe Experience Platform

The Opportunity From the moment you wake up until you go to bed, chances are that most of the media, ads, apps, websites and online shopping you interact with were touched by an Adobe product. Adobe Experience Platform (AEP) is the backbone that powers these experiences: it helps enterprises such as Verizon, Nike, AT&T and Disney centralize and standardize their customer data, build 360-degree customer profiles, and drive real-time personalized experiences. AEP processes terabytes to petabytes of data and serves billions of requests every day. We are expanding our Compute / Data Distiller team in Bucharest. The team owns two core AEP capabilities: • Data Distiller – a SQL engine on top of the AEP data lake that lets customers run interactive, batch and scheduled queries, accelerated queries, templates and alerts, and connect BI tools through the PostgreSQL protocol, to derive datasets, audiences and insights. We are also building AI agents that generate and personalize SQL from natural language. • Compute Gateway – the multi-cloud, multi-engine compute layer that orchestrates data processing workloads with policy-based routing and support for long-running applications and jobs. This position is for a key contributor who will help design modern data architectures and highly distributed cloud solutions, and take them all the way to production. What You'll Do • Design, build and operate complex features in a large-scale, multi-cloud, microservices-based SaaS product. • Own the outcome: lead application, framework and service design efforts, optimizing for quality, testability, scalability and performance. • Work on query execution, Spark SQL and query optimization, and the orchestration of compute workloads across multiple engines and clouds. • Prototype new approaches and deliver production solutions at scale; research new technologies and keep up with industry trends. • Partner with Product Management, Design, Data Science and engineering teams across domains, including our US counterparts on a shared roadmap. • Provide guidance in architecture, design and code reviews; mentor teammates and promote good engineering practices and continuous learning. • Take part in on-call activities on a follow-the-sun rotation with the rest of the team. • Contribute to open-source projects where relevant. What You Need to Succeed • B.S. in Computer Science or equivalent engineering degree, and/or 3+ years of relevant experience building software systems. • Strong programming skills in Java and/or Scala, with a solid grasp of OOP, functional programming and design patterns. • Proven expertise with distributed data processing: Apache Spark (including Spark SQL), Databricks, Delta Lake or equivalent Lakehouse technologies. • Good understanding of SQL, query engines and query optimization, table formats and schema evolution. • Hands-on experience with performance optimization, large-scale cluster tuning and efficient big data architectures. • Experience building scalable, resilient backend services and REST APIs. • Experience with cloud platforms (AWS and/or Azure). • Demonstrated experience delivering and operating a high-quality SaaS solution: testing, CI/CD, monitoring, infrastructure as code. • Experience implementing observability best practices (monitoring, logging, tracing) in distributed systems. …