At IBM Mexico Client Innovation Center, We are a leading consulting organization seeking a highly skilled and motivated Application Developer to join our dynamic team. This role offers a unique opportunity to leverage your expertise in cloud technologies to develop scalable, robust, and high-performance applications for our diverse clientele. If you are a passionate technologist who enjoys working in a fast-paced environment, thrives on challenges, and is ready to make a significant impact on our clients' businesses, we would love to hear from you. We offer a competitive salary and benefits package, opportunities for professional growth, and a supportive work environment.
Join us and help shape the future of cloud computing!
Apply today to become a key player in our Application Development team.
Our commitment to diversity and inclusion ensures that every voice is heard, respected, and valued. We encourage candidates from all backgrounds to apply. IBM is an Equal Opportunity Employer, committed to fostering a diverse and inclusive environment.
In this role, you'll work in one of our IBM Consulting Client Innovation Centers (Delivery Centers), where we deliver deep technical and industry expertise to a wide range of public and private sector clients around the world. Our delivery centers offer our clients locally based skills and technical expertise to drive innovation and adoption of new technology.
Phase 1: Job Assessment and Code Preparation
1. Job Selection and Inventory
· Select target jobs from EMR JIRA project for migration prioritization
· Identify and catalog all jobs eligible for EMR Serverless migration
2. Code Discovery and Validation
· Locate jobs in production environment and retrieve latest code versions
· Verify job status to ensure active (non-retired) jobs are migrated
· Audit code repositories to confirm production code alignment with master branch
3. Source Code Management
· Push latest production code to master branch if discrepancies exist
Phase 2: Infrastructure and Compatibility Updates
4. Storage and Query Migration
· Migrate Hadoop and HDFS commands to utilize Amazon S3 for data storage
· Update data processing workflows to leverage Amazon Athena for query execution
5. Runtime Environment Modernization
· Update Spark version to EMR Serverless compatible releases
· Upgrade Python version to supported EMR Serverless specifications
· Ensure all dependencies are compatible with target environment
6. Logging and Monitoring Integration
· Implement native EMR logging integration with AWS CloudWatch
· Configure AWS Log Groups for centralized log management
· Optimize scripts for enhanced observability in serverless environment
Phase 3: Testing and Orchestration
7. Documentation and Workflow Validation
· Create/update test documentation for migrated jobs
· Verify existing workflow configurations and dependencies
8. Job Orchestration Migration
· Identify current job triggers (Control-M, cron, etc.)
· Migrate Control-M scheduled jobs to AWS Step Functions
· Configure serverless-compatible job orchestration
Phase 4: Deployment and Validation
9. Non-Production Deployment
· Deploy migrated jobs to non-production environment
· Conduct internal testing and validation
· Coordinate handoff to AMI team for additional validation
10. Quality Assurance Testing
· Deploy to QA environment for comprehensive testing
· Conduct internal testing and validation
· Obtain AMI team sign-off on QA results
11. Production Release
· Create production release change following AMI team approval
· Execute controlled production deployment
· Implement post-deployment monitoring and validation for 1 week
· Proficient in Spark job development and optimization, including version upgrades and compatibility with EMR Serverless.
· Strong understanding of AWS services such as EMR Serverless, S3, Athena, CloudWatch, and Step Functions.
· Advanced skills in Python, including upgrading and refactoring code to meet EMR Serverless runtime requirements.
· Experience with job scheduling tools (e.g., Control-M, cron) and migrating workflows to AWS Step Functions.
· Proficiency in Git-based workflows, including branch management, code auditing, and repository alignment.
· Ability to integrate and configure logging solutions using AWS CloudWatch and Log Groups for serverless environments.
· Skilled in creating test documentation, conducting QA, and validating job performance across environments (non-prod, QA, production).