The project will explore the potentially transformative value of Foundation Models (FM) and generative AI (GenAI) approaches for scientific discovery. The ability of such approaches to handle vast amounts of data and learn complex patterns makes them particularly promising in physics for providing insights that conventional methods may miss. The last few years have seen the emergence of AI emulators trained on data alone that rival the performance of HPC (High Performance Computing) simulations on complex real world systems such as atmospheric dynamics. At the same time, there is plenty of active research that emulates textbook PDEs (Partial Differential Equations) to high accuracy. Finally, there are well known advancements in generative models for vision, time series and related modalities. The student will focus on building bridges between these domains exploring potential for continuous enhancement and development of multi-modal FMs for transfer learning and emergence of new skills.
The student will have access to rich datasets from cutting-edge facilities as well as IBM's compute infrastructure.
We offer a fully-funded 4-year PhD position at the intersection between computer systems design and AI as part of a new initiative between IBM Research Europe and Trinity College Dublin (TCD). The PhD project will be jointly supervised by Professor John Kelleher ([1] https://www.tcd.ie/scss/people/academic-staff/kellehj3/)
The student selected will be employed by IBM for the duration of the PhD and be a registered student at TCD with the following benefits:
* Access to resources and expertise both at IBM Research and TCD
* Research experience in both private and public sectors
* A substantial PhD Salary (>40,000 euro per annum)
* Full TCD PhD program fees (EU or non-EU level) paid
References
Visible links
1. https://www.tcd.ie/scss/people/academic-staff/kellehj3/
The following research questions will be tackled by the student: Can existing vision or time series models be used for scientific purposes? Can models trained on textbook PDEs be used for complex real world applications? How can we know when to trust the output of generative models applied to such domains? The prime application of the developed approaches would be for modelling plasma fusion for power generation and/or heliophysics.
A few examples of relevant references:
* 4M: Massively Multimodal Masked Modeling [1] https://arxiv.org/pdf/2312.06647
* Foundation Models for Time Series Analysis: A Tutorial and Survey [2] https://arxiv.org/pdf/2403.14735
* Poseidon: Efficient Foundation Models for PDEs [3] https://arxiv.org/abs/2405.19101
* Predicting Disruptive Instabilities in Controlled Fusion Plasmas Through Deep Learning [4] https://www.nature.com/articles/s41586-019-1116-4
As part of this PhD program your responsibility will include: fulfilling you obligation as registered student of TCD, performing research work (state of the art evaluation, devising novel solutions, etc.), writing research papers, interacting with international teams at IBM and TCD, and prototyping of ideas for evaluation and demonstration purposes.
References
Visible links
1. https://arxiv.org/pdf/2312.06647
2. https://arxiv.org/pdf/2403.14735
3. https://arxiv.org/abs/2405.19101
4. https://www.nature.com/articles/s41586-019-1116-4
- An honored degree in computer science, applied mathematics, physics or similar.
- Basics of Machine Learning and AI: knowledge of ML fundamentals and common Deep Learning architectures; experience with AI/ML libraries such as Pytorch.
- Basics of Mathematical Physics: mathematical fundamentals, familiarity with PDEs.
- Good programming skills: Python, version control, ability to write complex code from scratch and extend existing codes.
- Strong interest in performing basic research work, including: evaluation of existing literature, ideation of possible/alternative solutions, writing research papers.
- Ability to move quickly from idea to software prototype for evaluation and demonstration.
- Excellent spoken and written English.
- A background in computational and/or data-driven methods.
- Familiarity with numerical methods and/or Deep Learning methods for solving PDEs.
- Knowledge of Foundation Models architectures with a specific focus on multi-modality, Time Series and Vision.
- Knowledge of GenAI approaches: model architectures, fine-tuning, transfer learning.
- Advanced Programming skills: Python, data preprocessing (pandas, xarray), experience with HPC and Cloud environments.
- Track record of published academic papers.