When I was preparing this newsletter edition, unimaginable heat waves and devastating flash floods were once again in the news. These and other unfortunate weather phenomena repeated across the globe year after year prompted me to look at what's happening at the interface between data, AI and climate and weather sciences. This month’s Dev Breakfast is focused on inspiring and cautionary tales of data and AI in this ever-changing data-driven heating up world.
Meet the curator of the month
Ksenia is a Data Engineer with a background in atmosphere physics. Despite this career shift, she has always worked with data and loved to crunch numbers and solve complex problems. She describes herself as a “good old data engineer soldier” and enjoys building classic ETL/ELT pipelines, orchestration, and data modelling as well as social and communal aspects of working in diverse tech teams. Ksenia continues following environmental sciences from the sidelines. In her free time, she likes to cycle, work with wood, and read history books.
DestinE is an ambitious EU initiative to develop Earth’s highly accurate digital twins to aid adaptation and mitigation efforts. Groundbreaking, innovative, pushing the limits, unique - this webportal doesn’t spare the praise when describing three core components of DestineE - Digital Twins themselves, Digital Twins Engine (software behind digital twins), Data Lake (built for data discovery, processing and access), and Data Platform (entry point for users and applications).
There is a lot to discover on this portal - you can find links to each component’s own portals, documentation, explainers and case studies - I don’t want to limit your curiosity. Personally, I found the most interesting Digital Twins Engine explainer and how they approach dealing with mind-boggling petabyte-scale data amounts.
Accurate weather prediction is critical to all spheres of our life but it’s costly - computational models require supercomputers, large software and big teams of experts. There are many initiatives across the planet to make this process faster and/or cheaper, using ML and AI methods, but I want to share this one with you that aims to replace the typical end-to-end weather forecast creation pipeline. How fantastic would it be if one could run a weather prediction pipeline on a desktop computer? Have a look at the high-level description of the Aardvark system, and if you feel inspired - navigate to the Nature journal paper they linked at the bottom of the page.
Hot tip on how to read Nature papers: all more general content is at the beginning, and all technical details are at the end or even in supplements.
This article does an outstanding job of presenting in much detail how the GenAI revolution changed energy demands by datacenters, and how not-straightforward it is to understand personal impact, not lastly because cloud providers don’t disclose numbers on how much energy is used for GenAI models. One thing for sure - it’s important to be informed on the topic as GenAI has become a daily fixture in our work and personal lives.
Hot tip: check out more stories they have written on the topic in the series “Power Hungry: AI and our energy future”.
Tell us about one project you’ve worked on that you found particularly interesting or challenging.
For the last two years, I have been part of a long-term project to help build a big data platform for one of the Finnish household name companies. It’s truly a multidisciplinary, multi-vendor affair that requires lots of soft skills, particularly listening and communicating. Every day brings new tech tasks that require getting to know stakeholders, their needs, worries, but most of all their invaluable expertise and domain knowledge. This project enhanced my belief that tech skills take you far, but soft skills will take you even further.
Is there a hobby or interest that you’re passionate about?
I discovered woodworking several years ago and find it to be incredibly relaxing and meditative. In the workshop, when you hold a piece of wood in your hands, the outside world temporarily disappears. I believe that if your job involves being in front of the screen all day, working with your hands stimulates the dormant parts of the brain. Win-win situation!
Any tips for maintaining a healthy work-life balance?
Remember to reflect regularly on your well-being. This will require you to be honest with yourself and be able to admit “yeah, I am not OK” if you are in fact not doing OK. It’s easy to be busy and ignore the signs of overworking or burning out. If you are mindful and aware, you will have higher chances to maintain this magical work-life balance that keeps you mentally stimulated and challenged just the right amount.
Join us!
Senior data engineer
Come work with us if you are interested in becoming a trusted advisor for our varied customers. We strive to deliver state-of-the-art data engineering solutions and deliver strong business outcomes, while investing in our experts’ growth. This is a safe space to be yourself, develop skills you want to develop, and reach your career goals.
