By: Michael Feldman

The Blue Brain Project has deployed its fifth-generation supercomputer to support its mission of simulating an entire mammalian brain by 2020.

The 13-year-old project is being funded primarily by the Swiss government and is based in École polytechnique fédérale de Lausanne (EPFL), a premier research institute and university in Switzerland. In a nutshell, the project’s goal is to digitally reconstruct and simulate the mammalian brain, starting with a mouse model and eventually moving on to humans. Blue Brain 5 is intended to be used only to reach the project’s nearer-term goal of modeling the entire region of the mouse brain by 2020.

Known as Blue Brain 5, the new system is an SGI 8600 machine built by Hewlett Packard Enterprise (HPE). By today’s supercomputing standards, the system is not a particularly large, delivering a peak performance of just over one petaflop from its 372 nodes. Memory capacity is a paltry 94 terabytes.

The system derives its computational horsepower from Intel Xeon Gold 6140 and Intel Xeon Phi 7230 processors, as well as NVIDIA Tesla V100 GPUs. The Xeon CPUs run at 2.3 GHz and are equipped with 18 cores apiece, while the Xeon Phi processors have a base frequency of 1.3 GHz and are outfitted with 64 cores. External storage consists of four petabytes of DataDirect Networks (DDN) gear, which includes an 80 GB/second Infinite Memory Engine (IME) flash-based burst buffer. The system is hooked together with Mellanox InfiniBand.

Blue Brain 5 will take over from its predecessor, an IBM Blue Gene/Q supercomputer installed in 2013. That machine topped out at 839 peak teraflops – just a little less powerful, floating-point-wise, than its 2018 replacement.  In the latest TOP500 rankings, this IBM system anchored the bottom of the list at number 500.

The reason for the rather modest floating-point capacity of Blue Brain 5 is that flops are not the driving metric for brain simulations – memory bandwidth and I/O performance being much more important. As Blue Brain Project director Henry Markram explained: “Simulating the human brain is a big data challenge.”

That said, it will still take a fair amount of computational muscle to perform such simulations. According to Felix Schürmann, Co-Director, Blue Brain Project, the work is ultimately limited by the capacity of the supercomputer hardware. “Modeling an individual neuron at Blue Brain today leads to around 20,000 ordinary differential equations – when modeling entire brain regions, this quickly raises to 100 billion equations that have to be solved concurrently,” explained Schürmann.

The simulations encompass different biological scales, starting at the molecular level of genes and proteins and moving up to more complex structures of synapses, neurons, axons, and glial cells. The idea is to build a representation of the microcircuitry of the brain, such that sensory inputs like touch or taste can be modeled and tracked.

Ultimately, the project wants to model cognition and how different cognitive states are mapped to this microcircuitry. The models are also expected to be useful in medical research for conditions like Alzheimer’s and traumatic brain injuries.

Experimental data provides some of the framework for these models, but it can’t provide anything close to the detail being sought. At this point, the project’s latest digital reconstruction of the cortex microcircuitry is derived from experimental data that represents less than one percent of the synaptic connections. The challenge of the reconstruction process is it has to turn this sparse data into a faithful representation of the cellular anatomy and connectivity of the whole cortical structure.

Unlike previous iterations of supercomputers for the project, Blue Brain 5 has been crafted to more closely align with the workflows associated with the brain simulation work. To that end, the new supercomputer is comprised of four subsystems: one for extracting data from external storage, another that extracting data from memory, another for visualization, and a general-purpose subsytems that coordinates the activity of the other three.

Blue Brain 5 is installed at Swiss National Supercomputing Centre (CSCS) in Lugano, Switzerland. It went into production in April.

Image: Basket cell in a digital reconstruction of neocortical microcircuitry. Source: Blue Brain Project