Programming languages have been evolving rapidly over the years. The parallel with the evolution of popular music is quite stark. The real great oldies C and C++, like the Beatles, Pink Floyd, The Doors and Dire Straits, have lasted many years and are still going strong. Java was the next generation, like The Traveling Wilburys, R.E.M., Guns N’ Roses and Bon Jovi, now old too, but still very popular.
The new millennium has been, for the most part, nondescript. Programming languages fragmented, as did pop music, into smaller trends. This was in alignment with software development methodologies in practice today, such as agile and rapid prototyping, catering to the fast and short development cycles required by business today. And well aligned with the fast paced evolution of application platforms and usage patterns with the proliferation of mobile apps and the increased used of the web to manage most if not all life’s activities.
One significant change is that we now have programming environments, not just languages.
Over the last six months, I’ve been conceptualizing various ideas, primarily with the intent to build a prototype application that covers a wide gamut of programming environments, all working together to provide the complete solution. I ended up developing a complete IoT prototype, with sensors connected to an Arduino board that communicated over WiFi with a Raspberry Pi running an MQTT server, with a cross-platform mobile app that called an API to display the data and to store it in a NoSQL database.
Note:
This is a relatively old article and while surprisingly most tecnologies in the list are still going strong, there may be some that may not be relevant any more and there will new ones added.
The List
The list that follows has brief descriptions of the programming environments I used. It is not a comprehensive list by any means, neither is it necessarily the most recommended. And sadly, none of the environments can be guaranteed to be relevant even a year in the future.
The listing is not in any particular order, but attempts to follow the front end, middleware and back-end sequence.
- Material UI and Bootstrap
HTML design frameworks with improved design for mobile apps. Design amateurs can design pretty cool looking, although boiler-plate user interfaces.
- AngularJS and ReactJS
An significant enhancement to jQuery with many advantages, such as two-way data binding and a templating engine built-in. These two frameworks appear to be the choice of environments going forward, especially for mobile apps, which use the Single Page Application (SPA) design pattern.
- Processing
Processing is a flexible software sketchbook and a language for learning how to code within the context of the visual arts. Basically this means the software allows for very easy programming for visual representations of data, including charts and animations. This is a great option for working with Arduino circuits since data from sensors can be fed to the Processing software for visualization.
- Cordova/PhoneGap
One of my favorites since it allows you to develop a single code-base in HTML5 and JavaScript and build applications for multiple environments. Although purists may argue that you lose some of the native user experience, for applications where that is not very important, the saving in time, effort and cost is tremendous.
- MQTT
Message Queue Telemetry Transport is a lightweight messaging protocol for use on top of the TCP/IP protocol. It is just another message broker protocol with a publish-subscribe messaging pattern for distributing messages to interested clients based on the topic of a message. The difference is that it is designed for connections with remote locations where a small code footprint is required or the network bandwidth is limited, thus making it ideal for an IoT solution.
- NodeJS / Express
Server-side JavaScript keeps programming simple, just like client-side Javascript. And makes exchanging data via JSON objects a breeze. A routing middleware like Express makes writing APIs so simple, it’s hard to believe.
- MongoDB /Mongoose
For most of us who have grown up using relational databases, thinking of data structures in any format other than rows and tables is a huge effort. Once you are able to achieve the change to thinking about data structures as collections (tables), fields (columns) and documents (rows) a NoSQL database actually is very convenient to use. You can do almost everything you would want to in a relational databases, with maybe a few exceptions, which require application level interference. The flexible schemas (you are not stuck with the columns defined when the database was created) is of incredible value to businesses that are dynamic ad adding new functionality.
And documents are BSON objects, which are similar to JSON. So that’s one format across the front-end, middle-ware and database. Huge!
- Swagger
Swagger is a simple yet powerful representation of RESTful API. With a Swagger-enabled API, you get interactive documentation, client SDK generation and discoverability.
This projects took me a little beyond the realm of software, so concluding with a list of innovative hardware solutions:
- Raspberry Pi
The Raspberry Pi is a series of credit card-sized single-board computers developed in the United Kingdom by the Raspberry Pi Foundation to promote the teaching of basic computer science in schools and developing countries. These can act as complete, albeit low-powered, computers and interface very well with Arduino electronic projects.
- Arduino
Arduino is an open-source project that created microcontroller-based kits for building digital devices and interactive objects that can sense and control physical devices. These kits and the compatible sensors make building electronic projects easier than baking a cake.
There were a few notable software components that I evaluated but did not find suitable use cases for to incorporate them into any of my projects.
- Redis
Redis is an in-memory data store. Works similar to a NoSQL database but stores all data in the RAM thus obviously speeding up the processing by orders of magnitude. This is a solution that could be useful in production for the performance benefits but did not offer anything for just a prototype.
- RabbitMQ / Apache Kafka
Both are message brokers used primarily for asynchronous data streaming between producers and consumers where consumers can be notified when messages are available rather than needing to poll a data store. Message queues are designed for consumers that are equal, that is, any consumer may consume a message and process it. Targeting messages to a particular consumer is possible but not a natural design. I ended up using the findOneAndDelete option in MongoDB as an alternative to a non-persistent message broker.
In Conclusion
My experience at the end of this project was that some of the new technologies abstract the underlying complexities very well and make programming very simple and quick. Others have actually added a whole level of complexity and while I am sure at a deep level they add value, at first glance they appear daunting. And finally, all of them are not very stable, in the sense that they keep adding and changing and sometimes even removing, features across versions. This is not a great idea when using these technologies for a production system, which may be well advised to stick with the tried and tested technology as retro as it may seem.
In all my experimentation, the product (or service really) that made the biggest difference is a relatively old offering - the Infrastructure as a Service (Iaas) from Amazon Web Services (AWS). Access to pay-per-use, server-grade hardware on demand allows so much more processing power at a very low cost, taking away the need to invest in expensive hardware.