Data Infrastructures for the rest of us - III - software

Abstract: In part I, we highlighted how memory and CPU requirements impact your data infrastructure, your data engineering efforts and how the right algorithm and the right technology makes or break a data science project. In part II, we continued on hardware and caching, and mentioned ZFS and also the ZFS survival guide. In this latest installment, we briefly touch on virtualization with LXD and go on to install some really useful software to have in your data infrastructure, like PostgreSQL.
It's All Virtual Why do people like cloud computing? No, it's not the price. Most developer give no attention to questions of operating costs, ROI, support etc. What they do care about is getting a brand new "server", whenever they need one. Sure, the server might only have 2 CPU cores and 4GB of RAM, but all they had to do is click on provision, or run a command line tool, and their server is up and running. Then it is time to experiment. Get onto the new server, try things, m…

Democratization of Computer Vision

We need answers! We have data!

Signing data with Signethic

Linux Mint 19.3 installation with ZFS root

Dion Research provides Data Quality Suite to Freddie Mac

Data Infrastructures: ZFS survival guide