Keeping up with High Performance Computing’s ever-expanding horizon, with a chance of clouds. ☁️

Keeping up with High Performance Computing’s ever-expanding horizon, with a chance of clouds. ☁️

Keeping up with High Performance Computing’s ever-expanding horizon, with a chance of clouds. ☁️

On the 50th Anniversary of Cray Research Inc.’s founding by Seymour Cray, I thought I’d summarise my brief but intensely interesting experience with the last crop of Cray Supercomputer systems.

In Robinhood NoSQL I experimented with porting the Robinhood database from Elasticsearch, it being a commonly-used NoSQL database with specialisation in behaving like a Search Engine. Having ingested the ENTRIES table I now need to augment the records with filepaths from ANNEX_INFO, as I found out previously:

Robinhood provides indexing, search and management capabilities to computing centres with massive high-performnace filesystems, namely Lustre.

It does this by scanning every file on the filesystem as efficiently as possible to initially populate its MySQL/MariaDB relational database, then it continuously listen for changelogs coming from Lustre to keep its state in-line with reality.

The Lustre community supports recent point-releases of RHEL/CentOS but those patches don’t seem to come back into existing Lustre releases, since those releases don’t officially support later kernels. I installed client 2.10 thinking it had the best chance of working but encountered huge numbers of errors when compiling!