Data Engineer - Search Department (SED)

Job Description:

Business Overview

Rakuten Group is a global leader in internet services, empowering individuals and businesses around the world. With a diverse portfolio that includes e-commerce, fintech, digital content, and communications, Rakuten is dedicated to innovation and excellence. Our mission is to contribute to society by creating value through innovation and entrepreneurship. We are committed to leveraging cutting-edge technology to drive innovation and deliver exceptional value to our customers. We are looking for a talented Data Engineer to join our dynamic team and help us build and maintain robust data infrastructure.

Department Overview

The Search Department focuses on enhancing the Search, Discovery, and Navigation experiences for Rakuten users. We design, develop, and deploy high-performance, fault-tolerant distributed systems that are utilized daily by millions of Rakuten customers worldwide. Our commitment is to deliver innovative solutions that benefit both people and societies globally. Within this department, the Search Platform Engineering Group is responsible for delivering a robust search platform that serves a variety of services used by millions. This group plays a crucial role in ensuring that our search systems are not only efficient but also scalable to meet the growing demands of our diverse customer base.

Position:

Position Details
As a Data Engineer at Rakuten Group, you will be an integral part of our Search Data Platform team. Your primary responsibility will be to design, develop, and maintain scalable data pipelines and systems that support our search functionality. You will work closely with data scientists, analysts, and other stakeholders to ensure data is accessible, reliable, and optimized for improving search quality, relevance, and overall user experience. Your expertise in modern data technologies will be crucial in driving our data strategy forward and enhancing the performance of our search platform.

Key Responsibilities

Design, develop, and maintain scalable and efficient data pipelines using modern data technologies.

- Collaborate with data scientists, analysts, and other stakeholders to understand data requirements and deliver solutions that meet their needs.
- Implement and manage ETL (Extract, Transform, Load) processes to ensure data is accurately and efficiently processed.
- Optimize and maintain data storage solutions, including data lakes, data warehouses, and databases.
- Ensure data quality, integrity, and security across all data systems.
- Monitor and troubleshoot data pipeline performance and reliability issues.
- Stay up-to-date with the latest trends and best practices in data engineering and modern data technologies.
- Document data processes, systems, and workflows to ensure knowledge sharing and continuity.
- Contribute to the continuous improvement of search quality and relevance by providing reliable and well-structured data.

Mandatory Qualifications:

Bachelor’s or Master’s degree in Computer Science, Engineering, or a related field.

- Proven experience as a Data Engineer or in a similar role.
- Strong proficiency in SQL.
- Experience with modern data technologies such as Apache Spark, Apache Kafka, Apache Flink, Hadoop, and cloud-based data platforms (e.g., AWS, GCP, Azure).
- Proficiency in programming languages such as Python, Java, or Scala.
- Experience with ETL tools and frameworks (e.g., Apache NiFi, Talend, Airflow).
- Familiarity with data warehousing solutions (e.g., Snowflake, Redshift, BigQuery).
- Knowledge of data modeling, data architecture, and data governance principles.
- Strong problem-solving skills and attention to detail.
- Excellent communication and collaboration skills.

Desired Qualifications:

Experience with containerization and orchestration tools (e.g., Docker, Kubernetes).

- Knowledge of machine learning and data science workflows.

Knowledge of semantic search and image search related technologies.

- Familiarity with data visualization tools (e.g., Tableau, Power BI).

#engineer #applicationsengineer #aianddatadiv