Improve Code Quality with more Expressive, Powerful Languages

Software is eating the world in part because it leverages and enables powerful methods for automating tasks without re-inventing the wheel. This itself isn’t new. Operating systems have always provided a host of features and utilities that let applications communicate, manage files, allocate RAM and other resources, spawn threads and processes to perform tasks in parallel, and do other jobs.

These system-level capabilities are made more easily (and usually safely) accessible by programming languages. Popular languages with vast user bases and long histories (and sizeable corporate promoters), like C/C++ and Java, offer very complete standard libraries and avail users (one way or another) with wide choice of extensions for specialized computing. Java remains, of course, by far the most dominant language for enterprise computing.

Community-supported, open source programming languages have taken a while (on the order of decades) to catch up, and their evolution has sometimes been bumpy. The creators of Python (in the late 00s) made the difficult decision to shed technical debt: changing their languages in ways that broke much existing code in order to provide new features consistently. The need to maintain support for existing tooling caused Linux distro makers to standardize for years on Python 2, slowing adoption of Python 3 significantly. As a side effect, several generations of open cloud clients (including OpenStack’s, see below) and DevOps toolkits (e.g., Ansible) also standardized initially on Python 2, though support for Python 3 is now mostly in place.

Python is very much “batteries included” (always a part of Python’s mission statement) – richly endowed with built-in and community-enabled support for specialized data types (e.g., datetimes), protocols, Unicode, XML and JSON parsing, file and stream processing, and other oft-needed capabilities. Also built-in or easily installed from repositories are libraries, drivers, embedded clients and other infrastructure enabling convenient access to popular databases (e.g., MySQL, Cassandra, etc.); as well as domain-specialized extensions, modules, and libraries for mathematics, graphics, and other tasks.

The “batteries included” design pattern for programming languages has since been adopted by Go, introduced in 2009 by Google, open sourced, and in production release (version 1+) since 2012. Go is a compiled language inspired by C and C++, but simpler, with straightforward native support for light threadbased multiprocessing (GoRoutines), and robust error-handling by returned value (as opposed to throw/catch type mechanisms). These features make Go an excellent choice (say the language’s creators, ‘by design’) for building systems software on multiprocessor-equipped machines. Go is now also frequently used to create resilient, high-performance, cloud-oriented applications such as the container orchestration framework, Kubernetes, and Hashicorp’s template-driven deployment manager, Terraform. 

Python is the dominant language used in organizational DevOps automation (read: sophisticated deployment scripting), though PowerShell is becoming more popular in Windows environments. Python is also heavily dominant in data science and machine learning (among other things, it’s the most popular platform for working with TensorFlow, Google’s open source machine learning and numerical library). Google, however, has articulated misgivings about using TensorFlow with Python, due to the language’s dynamic typing and relatively slow speed. In late April, they open sourced Swift for TensorFlow on GitHub: a version of Apple’s compiled Swift language with first-class support for TensorFlow, that runs on Ubuntu and MacOS.

Web Projects

The documentary web (i.e., the web of ‘websites’) has long been dominated by content management system (CMS)-based architectures built around the so-called LAMP (Linux-Apache-MySQL-PHP) or LEMP (Linux-NGiNX-MySQL-PHP) stacks, which combine a webserver (Apache or NGiNX), the MySQL database, and a database-driven, PHP-based CMS such as WordPress, along with PHP for server-side processing. Once installed and hardened, these tools enable relatively easy website setup, design and updating using templates combining PHP, HTML, and tags enabling content insertion. CMSs support a rich plugin ecology for adding menus, search, dynamically-populated sidebar boxes, comment chains, and other simple programmatic elements to web pages. The venerable PHP language is easy to understand, flexible, provides rich capabilities, and is supported by an enormous userbase.

To add further client-side interactivity and create more-dynamic web applications, it’s now standard practice to use frameworks such as Angular.js, Backbone.js, or React.js. Written in javascript, these frameworks strive to eliminate the error-prone, complex process of creating dynamic web applications by using javascript to directly manipulate web page contents under the control of asynchronous communication (e.g., using XML or, more often, JSON datagrams) between page and application server. Instead, these so-called Model-View-Controller frameworks use an indirect approach in which front-end ‘views’ react to change events in the page, and update themselves by applying to a ‘model’ on the back end, with asynchronous communications managed by the framework, autonomously.

MVC frameworks are generally combined with Node.js/Express: an application server that executes javascript. They often use the MongoDB non-relational database, forming the so-called MEAN (MongoDB-Express-Angular (or other framework)-Node) stack.

Picking a Language for a Project

Choosing a language for a planned software project is rarely (if ever) purely a matter of comparing and selecting on the basis of language specifications, features, performance and other characteristics. Instead, the decision tends to be constrained in one of several ways.

In some cases, technology limits choice – for example, native Android applications are normally written in Java, with only limited support for essential C++.

Corporate IT policy may strongly favor use of a particular language. For example, Java, heavily used in enterprise server-side applications and mobile computing. Java has occupied the #1 spot on the TIOBE index of programming language popularity in 19 of the past 20 years.

In web environments, LAMP/LEMP or .NET are de-facto choices for classic websites. Choice of MVC frameworks is subject to frequent shifts in fashion (Backbone.js and React.js are becoming more popular for their relative simplicity; jQuery, darling of client-side developers before MVC took the spotlight, is poised to resurge with a more complete solution). Node.js is now increasingly popular as an application server for independent websites, though the JBOSS Java application server (Red Hat) remains a mainstay for large-scale enterprise websites. 

Where choice is not constrained by technology, policy, incumbency or other firm constraints, the language chosen for new projects will usually be the one with which a core team is most familiar. This is pragmatic: delivering software quickly means becoming productive quickly, and it can take a year or more to develop professional-level mastery of a new programming language.

Occasional exceptions to this pattern do, however, occur. Teams with longstanding familiarity with PHP and Java, for example, sometimes transition to Node.js/MEAN because they view end-to-end javascript as radically simpler than the LAMP/ LEMP multi-lingual (HTML/javascript on the front end, PHP on the back end, SQL for the database, etc.) model. Other teams seem to find compelling the fact that pre-packaged Node.js environments are well-supported by cloud providers in general, readily available in preconfigured containers, and supported as a preferred workload type on PaaS and serverless compute platforms.

Novel programming languages may also encourage teams to switch because they hold out the promise of radically improving programmer productivity, improving code correctness, or teaching coders new skills that are in-demand, hence monetizable. This may be the case with functional languages Haskell, Clojure, Erlang, Elixir, and Scala, all now growing in popularity. (In functional languages, programs are sets of functions that maintain no interior record of state, but merely work on their inputs and return results.)