Application Development Deep Optimizations

There are a lot of options for optimizing for speed – both on the client and the server side of the applications. I don’t think the list I’m going to give is or will ever be complete. Just – several ideas for starters to accelerate code execution. I’ve written about some of the options in the past: https://tomavelev.com/blog/Database%20Application%20Architectural%20Approaches:

  • Having some limited in size, but used a lot subset of the information in the RAM memory. May be valid for both the Client and the Server.
  • in RDBS – balance between the number of SQL queries with the amount of repetitive data transferred between the Database System and the Application Server
  • Use external better indexing tools – like Lucene for full text search, Cache Systems or maybe – sometimes files for complex, deeply linked, conditionally loaded, queried often data.

Cache Information on the Client Side – if your app is consumer/user/the broad public/oriented. More About in this article: https://tomavelev.com/blog/Types+of+User+Applications+or+Modules+of+Platforms+I+see+on+the+Internet. If you use cloud service like Firebase, the synchronization is done for you. If you need to code it yourself, it is in most cases done with several flags on the server and on the client:

  • Is Synchronized, Is Deleted, Date Modified
  • The logic around the flags – display only non-deleted records – apply deletion after synchronization
  • Check Date Modified for each item so to have only the latest version and a max(Date Modified) for taking delta changes – diff and pass around only the newer modifications.
  • Collisions may occur if some record is edited separately both on client and on the server or on several disconnected clients. There are several strategies to workaround this:
    • The simplest – to keep only the latest modification
    • Depending on the record type or the places of modification in them – merge the changes into a 3rd alteration – keeping them all
    • Have a duplicate records side by side showed to the user with naming signaling the device source/ date / version of modification and let him decide what to do.

There also an option to reload all records without additional flags – but it is the least optimal synchronization.

Understand, know and take into account the number of hits to the storage – even if you are delegating the information processing to outside of your programming code – system. Data gathered from the Application Server is redirected to Data /Database/ Storage System and is stupid to stop thinking about it just because “It is not in our code anymore”.

When hitting a relational database, many times, in a normalized data scheme requires from us to collect information from several tables. The database is optimized by pre-store sorting and indexes for fast searching, but if you use multiple joins, aggregate operations or even stored procedures – operations may slow down because of the multiple locations on the file system of the different type of records or runtime operations on the subset of records like sorting, filtering and so on.

Some noSQL databases workaround this visits to multiple files by minimizing normalization, creates data duplication and even destroys data integrity but brings the advantage of super speed because, if defined and stored correctly – with a single hit to the data storage system an application server will extract what it needs.

I talked recently with a colleague that opened my eyes for the Graph Databases. They try to keep the data structure and the references between records like the normal relational database, but also keep up with the speed of noSQL system. It also tries to store the semantic meaning of the relationship between records – not just “foreign keys”, or data references. According to my personal observations – the benefits of so big layer abstraction will be collected, acquired, achieved after enough individuals understand it and also link it to the real world events, dynamics and patterns in society and nature.

Stateless vs Statefull – Application Server, Database Systems and Applications in General. Today – the world is transitioning more and more to a stateless systems and architectures that benefits especially vertical scaling – adding more hardware running the same software stack and so – able to serve more and more clients. Relations between records is somehow state-full meta information so it is no coincidence that noSQL systems are easier to scale than the relational. Application Servers that bridge the clients with the data – do some logic with it and so on, are also getting stateless – multiplying more instances is just adding another docker or VM container. All of this is possible because the user and session information is shifting to the client – with cryptographic generated temporary tokens, that in the same time have the possibility to store some session, user information – minimizing the load and shifting the state from the App Server to the client.

Compiled programming languages, frameworks, libraries, approaches. Obviously – creating code closer to the iron is creating speed of execution. That’s why C++ is still active and there are several new languages – Go, Rust and more. The same getting lower concept has brought the compiler architectures everywhere – WebAssembly, Swift is LLVM, Dart in Flutter translates to binary thanks to the LLVM nature of SKIA library etc. The interpreted languages will not disappear, because there is a need for speed of development, but, especially if things hit scale – the optimizations are important.

Blocking vs Non-blocking – another tendency in last years is creating non-blocking architectures. Traditional Input/Output operations are Blocking and this limits the number of client connections. With non-blocking approach – the normal flow of an app is not blocked – waiting for some operation and the result is communicated back with a variable change or a call back that, in modern languages is pushed in a semaphore design pattern – names as “Stream”, Observer, Events, or something like that.

Share
Add comment