Data Engineer (m/f) - Porto

Detalhes da oferta

responsabilidades chave

• Identify?data?ingestion opportunities?that?unlock?insights, extract it and organize it in a way that’s widely available, scalable?and that answers any critical use case. You will also team up with data scientists to ensure?a constant feed of fresh data into machine learning models and support on the?organization of any performance related metrics and data;
• Support our clients in executing their Big Data strategies by designing and building?reusable components, frameworks and libraries at scale to support analytics products;?
• Design, build and launch extremely efficient and reliable data pipelines to move data across?several?platforms including?data?warehouses, data lakes, operational systems, online?services?and real-time systems;
• Anticipate, identify and solve issues concerning data management to improve data quality;?
• Organize and lead sharing sessions with peers, customers and functional partners and contribute to evangelizing data engineering practices;
• Guide and mentor?new?team members?;
• Collaborate closely with partners, strategy consultants and data scientists in a flat and agile organization where personal initiative is highly valued.

competências

• Proven experienceinsoftware development and practices with at least one programming language(e.g.Python, Java or C#);
• Proven experiencein SQL;
• Proven experienceanalyzingand modellingdatato discover opportunities and address gaps;
• Proven experienceinbuilding data pipelines within data ingestion projectsusing data flows management platforms(e.g.Airflow, Microsoft SSIS, Google Cloud Composer, AWS Step Functions, Azure Data Factory);
• Proven experiencewithcloud platforms(e.g.Azure, Google Cloud Platform or AWS Webservices);
• Proven experiencequerying massive datasets using Spark, Presto, Hive, Impala, etc.;
• Proven experiencewith more than one coding language;
• Proven experiencewith designing and implementing real-time pipelines;
• Proven experiencewith SQL performance tuning and E2E process optimization;
• Proven experiencewith anomaly/outlier detection;
• Proven experiencewith notebook-based Data Science workflow;
• Proven experienceto travel for business meetings, as required;
• Ability to work in the English language with strong written and oral communication skills.

principais benefícios