Speaking at the 2022 Energy High Performance Computing Conference (previously the Oil and Gas High Performance Computing Conference), BP distinguished geophysics advisor John Etgen set out to ‘tickle your complacency a bit’ by asking ‘are we approaching the singularity* in scientific computing’. The answer to the question is ‘no’. ‘We are actually going down the drain’.
Etgen is a geophysicist who has earned a ‘non trivial salary’ doing subsurface imagery for exploration, using math and physics to ‘see things that we have not seen before’. Etgen traced the history of high performance computing from his own experience, starting with running Fortran algorithms at school on a DEC PDP 10 and graduating to Amoco’s Cray 2 and the super-fast TMC Connection Machine. The latter was easy to code and ran 20x faster than the Cray. Since the CM (mid 1980s), there have been no ‘kick ass’ moments in HPC! Etgen’s current 25 petaflop cluster fails to provide the same ‘wow factor’. Today’s machines are cheaper, there are many more processors but the hardware forces us into an awkward compute model.
An earlier ‘singularity’ for humanity was the industrial revolution. This was characterized by the replacement of multi-tasking individuals by specialists. For Etgen, the next singularity will require even more specialization. While BP and others state that ‘we need cross-skilled, multi-talent hires’, this is ‘the fastest way to kill an industrial revolution type event’. We hear, ‘we need generalists’. Wrong! ‘If you want a singularity, let people hyperspecialize’.
Etgen was recently the editor of the Journal of Geophysics and realized that U-Net has ‘taken over the world’. The AI guys have got it. But if you are a hard’core computational physics person you don’t get it. Unless, that is you are using U-Net to replace conventional physics (something that Etgen personally hates!) Etgen regrets that there are no more ‘cowboys, cray, crazy lunatics’ like CM founder Danny Hillis. His crazy idea? Building a computing to match a problem in physics. (Hillis is currently building the Clock of the Long Now).
Going back to the industrial revolution, Etgen cited the legendary competition between steel driver John Henry and a steam-powered steel driving engine. Henry won against the machine … but died in the act! Today HPC faces exactly the same danger in the race between hand-coded finite difference stencils and AI-powered automatically-generated code. ‘Code that writes code that writes code…’ . Today the humans have the edge ‘but ultimately machines will mow over humans’.
Etgen wound up with a dig at the GPU HPC brigade. Learning CUDA is not the kind of specialization he is advocating. Again he returned to the CM and its concise Fortran Programming Guide of 1991 with ‘one page for every scientific algorithm’. The machine itself had one computational element for every grid point and could be treated like a big differential equation solver. Thanks to Hillis who made the machine look like your problem. Look at what HPC is doing today. Is it adding value? Hardware needs to be more abstractable, but ‘that is not the current trend’. And software too needs to be abstract, high level and re-usable. On the plus side ‘the AI guys are doing this already’.
In the Q&A Etgen was asked, ‘If you don’t do the GPU programming who will?’ He explained that he does some C++ coding ‘but this is not close to the metal’. This may be calling three levels of device drivers, below which, even more layers. ‘I just want to stand on the shoulders, just use the libraries – don’t write your own’.
* ‘Singularity’ refers to Ray Kurzweil’s 2005 oeuvre on artificial intelligence that envisaged a point in time when machine intelligence will be infinitely more powerful than human intelligence.
Comment – we take Etgen’s last comment to reflect the degree to which HPC has over the last couple of decades turned into a very specialized activity, that of tuning scientific code to ever more complex hardware. This is clearly not the kind of specialization that Etgen wants. But when he invokes AI-driven code generation, it is hard not to think of the complex hardware geometries that drive AI today (more of which in our report from the Nvidia GTC in our next issue). On the other hand, the AI brigade may be doing a better job of abstracting their logic into convenient Python libraries. Also, after the event we came across something that may be just what Etgen is looking for. In a report in First Break, quantum computers are presented as ‘replacing math by an actual physical optimization experiment’. We are not sure what it means but maybe some hyper-specialized crazy guy out there can pick this up and run with it.
More from the Rice HPC in Energy home page and also in our report elsewhere in this issue.
© Oil IT Journal - all rights reserved.