DevWorx - Blog

Easy XML to CSV Conversion

Countless tools, libraries and algorithms in use today assume inputs are already structured as rectangular or tabular data sets defined in terms of rows and columns. While the nomenclature can vary depending on the domain it all essentially refers to the same paradigm. For instance if you are using machine learning libraries, columns are often referred to as 'features' while rows are expected to represent 'observations'. Whatever the domain, the need to be able to...

GitHub Self-Hosted Raspberry Pi Build Farm - Setup and Costs

The Raspberry Pi 4 Model B, on top of being a captivating hobby and educational device, is actually quite a serious piece of hardware. As the cliché goes, only a few years ago a 1.5GHz quad core processor with 4GB of RAM was more than enough for everybody. And with a low retail cost and power consumption footprint it is easy to see why for basic computing tasks the Raspberry Pi can serve as a...

Apache Nifi Starter Project - Overcome Blank Canvas Inertia

Apache Nifi is a powerful addition to any developer’s toolkit. Creating and managing data processing pipelines are an almost inevitable fact of life when working on modern platforms. Recent trends in machine learning and data science are only reinforcing this paradigm. Don’t succumb to the temptation of developing your own data loading framework or pipeline - this comes with ongoing hidden costs as you continually end up having to reinvent solutions to typical data loading...

Beware - The Java ForkJoinPool in Web Applications.

The behaviour of Java’s ForkJoinPool is not always what it seems when you use it in a web application hosted in Apache Tomcat / Jetty and can lead to some surprising consequences. In this blog we explore some 'missing classpath resource' issues that can be caused by reliance on the ForkJoinPool Classloader in Web Applications. The Mystery of the Missing Classpath Resources Embedding resources in your classpath is a typical way to ensure that you...

Apache Spark on Java 11

Java 11 is the latest Long-Term-Support release version and all users are actively encouraged to upgrade. Hopefully, if you are still running Java 8 (or earlier) at this point, you have already started looking at some of the upgrade strategies available. There are excellent distributions available for you to choose from. Unfortunately, some of the libraries and frameworks I use on a daily basis (such as Apache Spark) is not quite ready yet as far...