Developer’s Life 21.03.2018 - ways of handling data

No matter what kind of application, game, tool, website or whatever you are developing, you probably need and use something to save stuff to. There are a lot of options. Each has it pros and cons. Each option has a specific case where it is best to be used and you should not create silver bullets in your head, so you not hit your foot.

The first option I’ll point out is writing stuff in file without some specific structure. In Java 8 and beyond you could write to file with one liner:
Files.write(Paths.get(new File(“fileToWriteTo”).toURI()), “Some String”.getBytes());

The second slightly structured way is by using key-value structured file or storage. There are a lot of APIs for this purpose in every environment – LocalStorage in JavaScript, Dictionary in .Net, ResourceBundle or Properties files in Java, pList files on the Apple Platform and so on.

Probably almost all the programming languages have some kind of API to store object or objects (object graph) to file. These APIs make saving and loading objects to and from file easy but there are some cons – most of this implementations are platform specific. For example you cannot easily save an object with Java and load it with Python or C#, except if you are using some standardized and widely used format – like JSON or XML. Third party file formats or way of saving files need to provide themselves APIs in every major language so that format could be usable.

The next most widely used way to save structured data are relational databases. The Database systems offer creating transactions – blocks of operations coupled together so either all the operations are applied or non of them (if some of them fail for some reason). This is one of the most important features. The pros of using old and standardized tools is that they are widely adopted. Every major programming language has an interface and even Object Relational Mapping Tool, Framework or API. There is cons of course that – they are not so easily scaled horizontally (cloned to a lot of machines) without a lot of overhead for propagating changes.

The next type of database are the Object-Orientated Databases. They provide APIs and Interfaces to the programming languages to pass and receive objects from them directly. They are usually faster than the relational databases. They are more developer-friendly as they need less code for storing and loading. The conses are they are mostly language locked and vendor locked. Their language relation makes them hard for managing from non-developers or developers with knowledge with other languages  – unlike SQL databases where you could swap the application server layer without much fuss. If some structure changes occur, SQL databases are more easily adaptable. With SQL you have full control on the data and you could execute complex transformations and load only slices of the data.

The next widely used database systems that solve the scaling problem and are really fast are the noSQL databases. In most cases they are key-value systems where the value could be any kind of data – Strings, numbers, object graphs and even whole binary documents. Their cons is – most of them lose the transactional isolation – you cannot group up several operation into one .

The database systems that provide the speed and scalability features of noSQL and the SQL Interface and structural organization of data are called newSQL. As the name says they are relatively new. They don’t provide tooling and interfaces as reach as the older ways of storing data. But they have their place in the market.

Over the years a hated the flame wars about different technologies. Every tool has a specific use case – pros and cons – areas where it does the job and where it doesn’t. Where these tools bring the most value and are more appropriate to use is for the developer to decide. What I have found over the years is that first you have to look to the end goal and from there – decide what is best. Elon Musk uses this approach. It is called First Principle. Of course from the beginning  – you cannot tell what tool will be more appropriate until you write it down, test it in production and adapt accordingly. Nothing could ever replace doing – the practice over the simple knowledge. This is true for every area of the life – not only for programming.

My Database Tool is mainly focused on the Relational Databases for now. If I see a need for myself or for someone else in the future I may add some more options for generation of code that is storing and loading the data with alternative ways and APIs.

Image provider :
Add comment