Will Python become the dominant programming language?

14ned

I saw the following on slashdot this morning: https://ask.slashdot.org/story/17/06/12/0136253/ask-slashdot-will-python-become-the-dominant-programming-language

In the PYPL index, which is based on Google searches and is supposed to be forward looking, the trend is unmistakable. Python is rising fast and Java and others are declining. Combine this with the fact that Python is now the most widely taught language in the universities. In fields such as data science and machine learning, Python is already dominating. "Python where you can, C++ where you must" enterprises are following suit too, especially in data science but for everything else from web development to general purpose computing...

People who complain that you can't build large scale systems without a compiler likely over-rely on the latter and are slaves to IDEs. If you write good unit tests and enforce Test Driven Development, the compiler becomes un-necessary and gets in the way. You are forced to provide too much information to it (also known as boilerplate) and can't quickly refactor code, which is necessary for quick iterations.

The original submission ends with a question: "Is Python going to dominate in the future?" Slashdot readers should have some interesting opinions on this. So leave your own thoughts in the comments. Will Python become the dominant programming language?

Language popularity index: https://pypl.github.io/PYPL.html. Note the constant exponential rise of Python for over a decade now.

Link about how Python is now the most taught language at universities: https://cacm.acm.org/blogs/blog-cacm/176450-python-is-now-the-most-popular-introductory-teaching-language-at-top-u-s-universities/fulltext

Putting my "rant hat" on temporarily, I find mixed feelings regarding this development. On the positive side, Python is far more valuable than Java ever was, Java in my opinion only ever excelled at one thing: it wasn't a truly awful choice of language for most problems, though it was almost never the best choice either, but as throughout the 2000s so few programmers were competent at any other language, you ended up with a lot of square Java pegs being hammered into round holes. Another positive is that Python is much, much more useful as a bridge between systems, mobile and web programming, and web programming is where an overwhelming majority of especially Irish graduates are nowadays trained at - for better or for worse (I being older would say for worse, but I recognise that despite my personal opinions on it, you don't really need to be able to program in assembler to be a competent engineer. Though I still think there a strong correlation between engineers competent at writing in assembler and competency in writing fast Javascript etc).

On the negative side, unlike Java, Python is a deep, deep, deep well of complexity and power. It approaches C++ in terms of just how complex and powerful it is, but does a far better job at hiding it than C++. That means that inexperienced Python programmers very frequently write lousy quality Python, and unlike C++ programmers, don't realise how awful their Python code is. The proliferation of graduates who can "program" in Python is going to very significantly worsen this problem as time progresses, indeed it may begin to threaten the Python ecosystem as the C++ ecosystem eventually was by mediocre code trashing good code through uncontrollable memory corruption, thus leading to disenchantment with the whole language.

Finally the other negative is that Ireland, as a nation, is still very much a Java major and .NET minor shop, and if the tech world is shifting over to Python as the next big thing, then Ireland is going to experience a major skills gap between what employers need and what's available in the talent pool. Having been out of contract for a while, I recently out of curiosity did a search for onsite roles around Ireland and I continue to be surprised at how early 2000s the roles advertised are here. So very different from the contracts I compete for on the global market. So for now I guess it's not a problem, but markets shift quickly, and before you know it we as a nation could be really struggling (again) with a large pool of unemployable graduates whilst employers have to import heavily from abroad to fill a skills gap. Not good.

Anyway, "rant hat" off. I'd be interested in what people think.

Niall

counterpointaud

Not sure where those numbers are coming from, but growth has not been exponential, and it is not the only language that has been growing in popularity.

https://insights.stackoverflow.com/survey/2017#technology-languages-over-time

Can't see JVM or .Net based languages going anywhere in the near future either.

DefinitelyMarc

I came here to mention your third point, but you already had it covered! I don't see the bigger companies shelving Java anytime soon, but Python certainly could make its own space if it eats in on Rails small business market. Java just has its teeth sunk too far into the stubborn Irish market in my opinion. It would be interesting if bootcamps took the place of the masters conversion courses that have taken over short-term taught programming though. That could boost Python adoption.

It's been almost a year since I touched Python, but one of the major problem I found was the divide between 2 and 3. It wasn't just a case of "I can't use this library" as I initially thought, we were constricted by some pretty important frameworks. Had to go with Python 2 to get Anaconda working on our locals and Flask. That's one of the biggest obstacles I can see to Pythons popularity. Despite this, it seems like Python is simply going to be the way forward. Companies ultimately hire what they can. Either that or a lot of Java developers are going to become more valuable for a while, before Java goes the way of the dodo.

14ned

counterpointaud wrote: »

Not sure where those numbers are coming from, but growth has not been exponential, and it is not the only language that has been growing in popularity.

https://insights.stackoverflow.com/survey/2017#technology-languages-over-time

https://pypl.github.io/PYPL.html is a measurement of total searches for learning tutorials in some programming language. It is therefore supposed to be a "leading" indicator instead of a "lagging" indicator like stackoverflow questions. The exponential growth graph is shown on their webpage towards the middle.

I would say that the exponential rise of Python matches my perception of global trends. Even in Ireland, there are startups and a few big banks crying out for experienced Python devs. But it's definitely nothing like as pronounced here as say in the UK.

Can't see JVM or .Net based languages going anywhere in the near future either.

Oh for sure. But then some US companies are actually hiring fresh graduates and training them in COBOL because of the worldwide shortage (i.e. read "expense") of COBOL programmers with tied five year contracts to make sure they don't run off. Me personally if I were a graduate I'd run a mile, but apparently they're paying +20% over normal graduate salaries they are that desperate. So technologies tend to linger on if there is a wealthy enough user base, even fifty years later. And it's not like COBOL has remained static, apparently it can speak UTF-8 now.

(I just did a search there to check COBOL use rates BTW, and was quite surprised to learn that Visual Studio 2015 has a COBOL programming plugin available. So you actually write, compile and debug COBOL from within the standard Visual Studio experience. They have a bang up to date development experience. I genuinely had supposed people were still stuck with ancient command line debugger or something, a silly assumption in hindsight)

Niall

counterpointaud

14ned wrote: »

https://pypl.github.io/PYPL.html is a measurement of total searches for learning tutorials in some programming language. It is therefore supposed to be a "leading" indicator instead of a "lagging" indicator like stackoverflow questions. The exponential growth graph is shown on their webpage towards the middle.

Interesting, thanks. I would not have expected Python to be more popular than JavaScript.

Still not sure I would call 8.7% growth from (~7% to ~16%) in five years 'exponential' though.

My guess is that the growth will level off, just like the growth in JS a few years back, and the growth in Ruby after Rails came out before that.

srsly78

After 20 years writing c++, now I work with scientific software and the holy trinity is c++/python/fortran. Rarely see java/.net used at all in this field. Only high performance stuff in c++/fortran of course. Python glues the whole lot together. NumPy is essential - without it Python would be useless for science/engineering. Also using python 2.7 anaconda here. Works fine on both windows/linux - but windows is second class citizen (needs special treatment).

Of course python has it's downside - the easy availability of packages turn you into a lazy bollix. Even worse, you get to the productisation stage and then lawyers make you go through every packages licensing etc to make sure no GPL is in there >.<

It's definitely the #1 language for non-developers (most scientists and engineers). And quite possibly #1 for lazy-developers as well.

No rants about whitespace yet?

counterpointaud

srsly78 wrote: »

No rants about whitespace yet?

I was restraining myself

Cordell

Too late now anyway.

Daledge

Currently a Computer Engineering student here just finished my third year. Throughout the course we've done three Java modules (Programming I, II and III) and are constantly being told Java is what employers are looking for. However when the job offers went up for placement (We've been on placement since the around January) it seemed EVERYWHERE had Python listed as a skill they were looking for.

Very little Java, so I would say from my very limited knowledge it appears to be way it's going.

Cordell

Because Python is used everywhere now, even if the main codebase is not Python, there is a fair change that it has Python bindings. There are countless examples of performance critical applications that have the core code written in very optimized C with assembly parts but with Python user interfaces and APIs. This is why it is a requirement everywhere now

14ned

counterpointaud wrote: »

Interesting, thanks. I would not have expected Python to be more popular than JavaScript.

Remember the linked page is web searches for tutorials in that language. So if nobody is looking for a tutorial in Javascript (likely), it will not appear to be popular relative to languages where people feel they need a tutorial because they are struggling.

Still not sure I would call 8.7% growth from (~7% to ~16%) in five years 'exponential' though.

You may have missed that the trend graph has a log scale on the vertical axis.

My guess is that the growth will level off, just like the growth in JS a few years back, and the growth in Ruby after Rails came out before that.

Well, I've been predicting that to my C++ colleagues since about 2001, and I keep being confounded - Python just keeps on exponentially growing in indicators for new programmer uptake. But you're right nothing can grow exponentially forever, it has to be a logistic curve, we're just waiting for the linear inflection point.

Niall

14ned

Daledge wrote: »

Currently a Computer Engineering student here just finished my third year. Throughout the course we've done three Java modules (Programming I, II and III) and are constantly being told Java is what employers are looking for. However when the job offers went up for placement (We've been on placement since the around January) it seemed EVERYWHERE had Python listed as a skill they were looking for.

Very little Java, so I would say from my very limited knowledge it appears to be way it's going.

It's definitely the case that Java is the most liquid developer role in Ireland today. It is the easiest to get hired in, also the easiest to get fired from because supply of Java developers is also the most liquid of any programming language, so you're relatively easy to substitute for.

Now, get a job in say COBOL in Ireland which would be very hard, chances are excellent they'll never fire you unless the business fails. Salaries tend to track difficulty in filling a role for the same geographical area by 10-20%, but rather than go past 15-20% they tend to leave the role unfilled and bleat loudly about "skills shortages" which are the fake made-up kind based on unwillingness to pay, rather than the real kind where you literally can't fill the role for any pay level (some top end finance roles are like this, a friend of mine in Finance in London seriously considered moving to Dublin until he figured out how much tax he'd pay, he'd actually earn more in Paris). Anyway, I won't get started on that tangent!

Niall

moycullen14

14ned wrote: »

It's definitely the case that Java is the most liquid developer role in Ireland today. It is the easiest to get hired in, also the easiest to get fired from because supply of Java developers is also the most liquid of any programming language, so you're relatively easy to substitute for.

Now, get a job in say COBOL in Ireland which would be very hard, chances are excellent they'll never fire you unless the business fails. Salaries tend to track difficulty in filling a role for the same geographical area by 10-20%, but rather than go past 15-20% they tend to leave the role unfilled and bleat loudly about "skills shortages" which are the fake made-up kind based on unwillingness to pay, rather than the real kind where you literally can't fill the role for any pay level (some top end finance roles are like this, a friend of mine in Finance in London seriously considered moving to Dublin until he figured out how much tax he'd pay, he'd actually earn more in Paris). Anyway, I won't get started on that tangent!

Niall

Interesting about managing the supply/demand curve for a particular skill.
Java has, for me, reached its sell-by date as a marketable skill.

True, there are no shortage of opportunities but the competition - especially at the junior level - is pushing rates down, or at least stopping them from rising.

Ideally demand for your skill set should be rising and - most importantly - supply should be lagging. I think we've passed that point with Java.

Couldn't agree more about 'unwillingness to pay' being presented as a skills shortage. Jack the rate up 20% and you won't be long seeing the shortages disappear.

Colonel Panic

High performance C++ backend code with Python bindings is where it's at for a large portion of the financial companies I've had interactions with so it's a nice little niche to get into.

14ned

Colonel Panic wrote: »

High performance C++ backend code with Python bindings is where it's at for a large portion of the financial companies I've had interactions with so it's a nice little niche to get into.

Yet generally speaking they won't employ remote working C++ specialists!

The funny thing with finance folk is either they pay ludicrous money to remote working C++ consultants for advice on a very limited (and usually vexing) problem domain, or else they want you onsite turning up 9 to 5. The whole notion of > 6 month longer term contracts with remote contractors appears to not fly in that industry for some reason. A shame, I'd take some of that action, I've only seen a month of income so far this year.

Niall

srsly78

That's the least of your worries. I see very few pure c++ roles around now - only see the "usual suspect" companies in dublin that have high turnover (everyone keeps quitting). Rates are also rubbish.

I got hired as a c++ developer in current job, but actually write very little of it! It's all about python now

14ned

srsly78 wrote: »

That's the least of your worries. I see very few pure c++ roles around now - only see the "usual suspect" companies in dublin that have high turnover (everyone keeps quitting). Rates are also rubbish.

I got hired as a c++ developer in current job, but actually write very little of it! It's all about python now

Heh, I was more thinking of London finance companies actually. I've never ever been employed in Ireland yet in nearly ten years of running my consulting firm. Irish employers don't have a culture of employing remote working C++ experts. Indeed, they won't even consider a two day onsite, three day offsite arrangement. It's either all onsite or nothing. I choose nothing, I live too far away from Dublin for all onsite to make any financial sense for me, if I ever needed go onsite due to continuing lack of remote work, I'd probably head to London and weekend it back home to Cork. I know I could clear £800-1000/day over there, and I'd probably need it given the price of B&B's near central London.

Niall

Pelvis

So am I making a bad decision choosing to focus on mastering Java/Spring/hibernate et all for my final year's studies?

srsly78

Noone cares what you studied in college (most of which will be either obsolete or give you bad habits). Just be careful about which jobs you pick - you might end up as a "java guy".

@ned: You are doing it wrong. When working in London kip on a mates couch, and pocket the subsistence allowance (e190 per overnight)

Colonel Panic

14ned wrote: »

Yet generally speaking they won't employ remote working C++ specialists!

The funny thing with finance folk is either they pay ludicrous money to remote working C++ consultants for advice on a very limited (and usually vexing) problem domain, or else they want you onsite turning up 9 to 5. The whole notion of > 6 month longer term contracts with remote contractors appears to not fly in that industry for some reason. A shame, I'd take some of that action, I've only seen a month of income so far this year.

Niall

There's a few remote contractors at my current role, but that DOES feel like an exception and they're very much in the "wunderkind" tier (and not C++ devs) but I'm a full time employee and happy enough with that for now.

Sephiroth_dude

Which is used more in the industry, python 2 or 3?

Pelvis

srsly78 wrote: »

Noone cares what you studied in college (most of which will be either obsolete or give you bad habits). Just be careful about which jobs you pick - you might end up as a "java guy".

But I'm hardly going to leave college and work as a python developer if I've my experience centers around Java? Or is it common for grads to get work using unfamiliar languages?

srsly78

It's no good just focusing on one language. You should be comfortable picking up new languages and frameworks. Also not good to focus on one domain either (e.g. frontend web development) - consider other areas such as graphics, maths, hpc, security, networking etc.

You could easily learn basic python on your own in a week. If you have a good degree and demonstrate enthusiasm I don't see why you wouldn't be hired. Consider that even in a "java shop" they probably use python scripts for various housekeeping stuff.

Pelvis

srsly78 wrote: »

It's no good just focusing on one language. You should be comfortable picking up new languages and frameworks. Also not good to focus on one domain either (e.g. frontend web development) - consider other areas such as graphics, maths, hpc, security, networking etc.

You could easily learn basic python on your own in a week. If you have a good degree and demonstrate enthusiasm I don't see why you wouldn't be hired. Consider that even in a "java shop" they probably use python scripts for various housekeeping stuff.

Yeah that's fair enough, the reason I focus on Java at the moment is because I know a little of a few languages, but not enough of any one, and that's been the case all through college. You start to "get" something and then you're moved on to something else. My thinking is, get to an advanced (as advanced as possible anyway) level at a current, relevant language, and that will be good grounding for if/when I decide to branch out to the likes of python, C++.

14ned

srsly78 wrote: »

@ned: You are doing it wrong. When working in London kip on a mates couch, and pocket the subsistence allowance (e190 per overnight)

Unfortunately I'm old enough that everyone I know near London has children, I couldn't really stay long. Besides, my accountant gets annoyed if I try pocketing the subsistence allowance without a "vattable receipt" as he'd put it.

Also, the Revenue allowances haven't been increased since 2009 or something. My last few trips abroad were well over the Revenue day rates, so ended up expensing everything. It was so convenient for a long time though, saved hassle with doing receipts. I hate doing receipts, boring, boring, boring.

Niall

14ned

Sephiroth_dude wrote: »

Which is used more in the industry, python 2 or 3?

The downloads for python 2.7 alone are several fold that for all the 3.x series combined.

That said, it's very obvious that most pypi packages now run on both. The future is clearly 3, all new Python code is written for 3 or 2and3. Runs a lot faster too with the same python, they've done a ton of efficiency improvements in the 3 series interpreter.

Niall

LincolnHawk

Java is going nowhere lads. You absolutely should have expert knowledge of it if you like money.
But you need to market yourself as a full stack developer. Add in a JS framework, say Angular 2 using typescript. Know your databases inside out, relational, noSQL. Stick with Spring and Hibernate.
Understand ESBs and Enterprise Integration patterns...

amen

still not sure I would call 8.7% growth from (~7% to ~16%) in five years 'exponential' though.

A little off post but a 7% growth to 16% is not a 8.7% growth its actually a 124% growth.

mahamageehad

I'm currently learning Python at the moment (to get into data analysis/machine learning) and it has the easiest intro curve of any language I've ever tried. If you've any understanding of programming at all, I reckon it's easy to make the move into Python. However, that said, I reckon that there's loads out there that never get *good* at Python.

14ned

mahamageehad wrote: »

I'm currently learning Python at the moment (to get into data analysis/machine learning) and it has the easiest intro curve of any language I've ever tried. If you've any understanding of programming at all, I reckon it's easy to make the move into Python. However, that said, I reckon that there's loads out there that never get *good* at Python.

The single most common problem with most Python is poor performance. A skilled Python programmer writes code which is very competitive with any other programming language in terms of scaling curve, indeed a long standing argument I've been having with the C++ gurus is that well written Python programs have better scalability than well written C++ programs. The absolute speed of C++ hides the lousy scalability up to a point.

The second most common problem is lack of portability. Too many Python devs only target some version of Python, usually 2.7. There are four major Python implementations (CPython, Jython, IronPython, PyPy (+stackless)), well written Python runs well on all four, and on both v2 and v3 language editions.

Niall

mahamageehad

Ah ok Niall, thanks for the extra info. I'm nowhere near that far along to actually understand all that just yet, but I've noted it for future reference. Currently I'm using Anaconda to separate out project environments so I can flip between Python 2.7 and 3.6 but I can see how it could be an issue in a live project.

14ned

mahamageehad wrote: »

Ah ok Niall, thanks for the extra info. I'm nowhere near that far along to actually understand all that just yet, but I've noted it for future reference. Currently I'm using Anaconda to separate out project environments so I can flip between Python 2.7 and 3.6 but I can see how it could be an issue in a live project.

When you do get a chance, look into "stackless Python". Very cool, though a poor relation to say Erlang.

Niall

Tar.Aldarion

Sephiroth_dude wrote: »

Which is used more in the industry, python 2 or 3?

Not sure but you will have a lot of companies (like where I work) that will use 2.7 and not cater for 3.x at all, for the foreseeable future.

As for the OP, I hope so, I love working with python.

srsly78

14ned wrote: »

When you do get a chance, look into "stackless Python". Very cool, though a poor relation to say Erlang.

Niall

Basically useless because the various 3rd party packages won't work with it. Similarly with pypy and others. They claim experimental support for things like numpy but it just doesn't work! CPython is the only game in town if you care about the ecosystem.

Talisman

I've gotten back into using Python recently and have noticed that some things that are supposedly the Pythonic way of doing things are not the most performant. I pointed out a few simple to fix performance bottlenecks in code and the response was along the lines of "performance isn't my concern here, it's correctness".

counterpointaud

amen wrote: »

A little off post but a 7% growth to 16% is not a 8.7% growth its actually a 124% growth.

You're right sorry, I meant difference between 7% of total and 16% of total, not % growth.

Colonel Panic

14ned wrote: »

The single most common problem with most Python is poor performance. A skilled Python programmer writes code which is very competitive with any other programming language in terms of scaling curve, indeed a long standing argument I've been having with the C++ gurus is that well written Python programs have better scalability than well written C++ programs. The absolute speed of C++ hides the lousy scalability up to a point.

The second most common problem is lack of portability. Too many Python devs only target some version of Python, usually 2.7. There are four major Python implementations (CPython, Jython, IronPython, PyPy (+stackless)), well written Python runs well on all four, and on both v2 and v3 language editions.

Niall

Are you on the Python not as scalable side?

14ned

Colonel Panic wrote: »

Are you on the Python not as scalable side?

Other way round. Mine would be a minority opinion in the C++ world, a majority opinion in the Python world. The Python guys are right.

People in the C++ world don't realise how poorly the C++ standard library scales. It was designed a very long time ago (> 25 years ago) back when you could consider malloc() to be free of cost and scalability didn't matter much because there was so little RAM anyway, 640Kb really was an ocean of RAM very few had access to. Python not only is much newer, but they've been systematically replacing internal algorithms to better fit modern hardware over time, so for example Python 3.2 or 3.3 (can't remember) replaced the old UTF-16 based implementation of unicode strings with a much faster UTF-8 one. Python code never noticed the difference, it just runs faster and scales better.

C++ can't match that, though there are increasing calls for an STL v2 based on the Ranges TS. The Ranges TS implements Python like list comprehensions and generators for C++, should be landing in the C++ 20 standard or near to it. The Coroutines TS also adds stackless C++ like stackless Python, so things are improving, just fifteen years later than everyone else.

I actually have a Google Summer of Code student looking into implementing the Ranges TS at compile time right now, the idea is you can run a ton of stuff in the compiler at compile time like looking up keys in maps, that way there is literally zero runtime overhead. C++ may be long behind the curve, but it gets there eventually.

Niall

Colonel Panic

Right, so when I think of scaling well with C++ code, I think about leveraging CPU cache by being good with data layout, custom allocators and task based parallelism, not the specifics of their respective standard libraries.

cnocbui

Yes and no.

Where speed of execution is critical, It isn't of much use.

14ned

Colonel Panic wrote: »

Right, so when I think of scaling well with C++ code, I think about leveraging CPU cache by being good with data layout, custom allocators and task based parallelism, not the specifics of their respective standard libraries.

That's exactly the problem. C++'s standard libraries cause poor data layout, have lousy custom allocation, and non-existent task based parallelism. Python's are much better on all three fronts. Hence why well written C++ usually scales poorer than well written Python.

Niall

Colonel Panic

I dunno, I mean I concede that the C++ standard library has issues and missing components, but that nature of the language grants you the ability to improve these things.

In Python, unless you're willing to write native implementations with Python bindings, I don't think you can call it more scalable, whatever that even means!

I believe in what I see, a hybrid future!

14ned

Colonel Panic wrote: »

I dunno, I mean I concede that the C++ standard library has issues and missing components, but that nature of the language grants you the ability to improve these things.

Dunno, me and my ilk have been pushing for STL v2 for many years. In fact I just finished writing an email just there to a WG21 member on the design for iostreams v2. I want STL iostreams taken out back and shot, zero backwards compatibility, and a radically more sensible design for iostreams v2.

In Python, unless you're willing to write native implementations with Python bindings, I don't think you can call it more scalable, whatever that even means!

Not true, you can reimplement Python core algorithms in Python with needing to drop to the C API. Python.Future (http://python-future.org/index.html) for example monkey patches in Python 3 replacements for Python 2 core objects like list, str, dict etc on a module by module basis. A fair chunk of that is pure Python. Then your Python 3 module can run on the Python 2 interpreter alongside Python 2 only code. Which is pretty cool.

Niall