Defining software is the next step in supercomputing for the Met Office

Obviously, this is an enormous amount of work and having superlative computing power is important. The Met Office began working with Microsoft on a £1.2 billion computing giant project last year, and has also begun investing in software-defined projects to balance productivity and protection.

Speaking at the Cybersecurity Festival of Computing last week, Ewen described the organization’s “Lab & Factory” approach.

There are all categories of use cases [for Met Office predictions], from the low-key but repetitive to the unique but very important, and in that we can’t afford to miss any more details. As an electronic and continuity challenge, this is very difficult, because on the one hand I need a very collaborative environment that will allow us to maintain our position as the world’s leading research institute in the field of weather and climate; On the other hand, you have a data factory that never misses a skip.”

Think of a supercomputer – in greatly simplified terms – as a “cool big Linux box”. Ewen said that partitioning in Linux is “difficult”, and therefore it is difficult to combine methods that meet the needs of the scientific community and are highly reliable.

Some of the benefits of partnering with Microsoft, as the business can be abstracted into the public cloud, is the ability to move to a configured software and service environment. “Things that I can’t do with the constraints of number of people, amount of resources, etc., can be done in the future – which will allow me to effectively offer a laboratory and factory on the same infrastructure, across configuration.”

Running both the laboratory and the plant on the same infrastructure brings huge benefits in workload migration.

Anyone who works in tech will be familiar with the concept of a dev stack, a pre-prod stack, and a production stack. The problem with that, in the past, was the ability to keep three stacks as isolated as each other and relevant to a production environment.

“My experience in the past has been that you’ll develop something in development, you’ll test it in pre-production and then you’ll push it right in; and for some unexpected reason, largely to do with the fact that the production environment isn’t fully representative of the production environment, something big went wrong.”

Working with superior cloud providers like Microsoft Azure and AWS makes it “very easy” to layer on security systems, which means migrating the workload becomes a risk.

The Met Office will continue to evolve the Lab & Factory approach over the next decade. To learn more, watch the video of Ewen’s conversation with Stuart Sumner from Computing below.

Leave a Reply

%d bloggers like this: