I have noticed that some applications or algorithms that are built on a programming language, say C++/Rust run faster or snappier than those built on say, Java/Node.js, running on the same machine. I have a few question regarding this:
I'd really appreciate if someone broke this down for me.
In programming language design and implementation, there is a large number of choices that can affect performance. I'll only mention a few.
Every language ultimately has to be run by executing machine code. A "compiled" language such as C++ is parsed, decoded, and translated to machine code only once, at compile-time. An "interpreted" language, if implemented in a direct way, is decoded at runtime, at every step, every time. That is, every time we run a statement, the intepreter has to check whether that is an if-then-else, or an assignment, etc. and act accordingly. This means that if we loop 100 times, we decode the same code 100 times, wasting time. Fortunately, interpreters often optimize this through e.g. a just-in-time compiling system. (More correctly, there's no such a thing as a "compiled" or "interpreted" language -- it is a property of the implementation, not of the language. Still, each language often has one widespread implementation, only.)
Different compilers/interpreters perform different optimizations.
If the language has automatic memory management, its implementation has to perform garbage collection. This has a runtime cost, but relieves the programmer from an error-prone task.
A language might be closer to the machine, allowing the expert programmer to micro-optimize everything and squeeze more performance out of the CPU. However, it is arguable if this is actually beneficial in practice, since most programmers do not really micro-optimize, and often a good higher level language can be optimized by the compiler better than what the average programmer would do. (However, sometimes being farther from the machine might have its benefits too! For instance, Haskell is extremely high level, but thanks to its design choices is able to feature very lightweight green threads.)
Static type checking can also help in optimization. In a dynamically typed, interpreted language, every time one computes x - y, the interpreter often has to check whether both x,y are numbers and (e.g.) raise an exception otherwise. This check can be skipped if types were already checked at compile time.
Some languages always report runtime errors in a sane way. If you write a[100] in Java where a has only 20 elements, you get a runtime exception. This requires a runtime check, but provides a much nicer semantics to the programmer than in C, where that would cause undefined behavior, meaning that the program might crash, overwrite some random data in memory, or even perform absolutely anything else (the ISO C standard poses no limits whatsoever).
However, keep in mind that, when evaluating a language, performance is not everything. Don't be obsessed about it. It is a common trap to try to micro-optimize everything, and yet fail to spot that an inefficient algorithm/data structure is being used. Knuth once said "premature optimization is the root of all evil".
Don't underestimate how hard it is to write a program right. Often, it can be better to choose a "slower" language which has a more human-friendly semantics. Further, if there are some specific performance critical parts, those can always be implemented in another language. Just as a reference, in the 2016 ICFP programming contest, these were the languages used by the winners:
1 700327 Unagi Java,C++,C#,PHP,Haskell
2 268752 天羽々斬 C++, Ruby, Python, Haskell, Java, JavaScript
3 243456 Cult of the Bound Variable C++, Standard ML, Python
None of them used a single language.
What governs the "speed" of a programming language?
There is no such thing as the "speed" of a programming language. There is only the speed of a particular program written by a particular progammer executed by a particular version of a particular implementation of a particular execution engine running within a particular environment.
There can be huge performance differences in running the same code written in the same language on the same machine using different implementations. Or even using different versions of the same implementation. For example, running the exact same ECMAScript benchmark on the exact same machine using a version of SpiderMonkey from 10 years ago vs a version from this year will probably yield a performance increase anywhere between 2×–5×, maybe even 10×. Does that then mean that ECMAScript is 2× faster than ECMAScript, because running the same program on the same machine is 2× faster with the newer implementation? That doesn't make sense.
Has this anything to do with memory management?
Not really.
Why does this happen?
Resources. Money. Microsoft probably employs more people making coffee for their compiler programmers than the entire PHP, Ruby, and Python community combined has people working on their VMs.
For more or less any feature of a programming language that impacts performance in some way, there is also a solution. For example, C (I'm using C here as a stand-in for a class of similar languages, some of which even existed before C) is not memory-safe, so that multiple C programs running at the same time can trample on each other's memory. So, we invent virtual memory, and make all C programs go through a layer of indirection so that they can pretend they are the only ones running on the machine. However, that is slow, and so, we invent the MMU, and implement virtual memory in hardware to speed it up.
But! Memory-safe languages don't need all that! Having virtual memory doesn't help them one bit. Actually, it's worse: not only does virtual memory not help memory-safe languages, virtual memory, even when implemented in hardware, still impacts performance. It can be especially harmful to the performance of garbage collectors (which is what a significant number of implementations of memory-safe languages use).
Another example: modern mainstream general purpose CPUs employ sophisticated tricks to reduce the frequency of cache misses. A lot of those tricks amount to trying to predict what code is going to be executed and what memory is going to be needed in the future. However, for languages with a high degree of runtime polymorphism (e.g. OO languages) it is really, really hard to predict those access patterns.
But, there is another way: the total cost of cache misses is the number of cache misses multiplied by the cost of an individual cache miss. Mainstream CPUs try to reduce the number of misses, but what if you could reduce the cost of an individual miss?
The Azul Vega-3 CPU was specifically designed for running virtualized JVMs, and it had a very powerful MMU with some specialized instructions for helping garbage collection and escape detection (the dynamic equivalent to static escape analysis) and powerful memory controllers, and the entire system could still make progress with over 20000 outstanding cache misses in flight. Unfortunately, like most language-specific CPUs, its design was simply out-spent and out-brute-forced by the "giants" Intel, AMD, IBM, and the likes.
The CPU architecture is just one example that has an impact on how easy or how hard it is to have a high-performance implementation of a language. A language like C, C++, D, Rust that is a good fit for the modern mainstream CPU programming model will be easier to make fast than a language that has to "fight" and circumvent the CPU, like Java, ECMAScript, Python, Ruby, PHP.
Really, it's all a question of money. If you spend equal amounts of money to develop a high-performance algorithm in ECMAScript, a high-performance implementation of ECMAScript, a high-performance operating system designed for ECMAScript, a high-performance CPU designed for ECMAScript as has been spent over the last decades to make C-like languages go fast, then you will likely see equal performance. It's just that, at this time, much more money has been spent making C-like languages fast than making ECMAScript-like languages fast, and the assumptions of C-like languages are baked into the entire stack from MMUs and CPUs to operating systems and virtual memory systems up to libraries and frameworks.
Personally, I am most familiar with Ruby (which is generally considered to be a "slow language"), so I will give two examples: the Hash class (one of the central data structures in Ruby, a key-value dictionary) in the Rubinius Ruby implementation is written in 100% pure Ruby, and it has about the same performance as the Hash class in YARV (the most widely-used implementation), which is written in C. And there is an image manipulation library written as a C extension for YARV, that also has a (slow) pure Ruby "fallback version" for implementations that don't support C which uses a ton of highly-dynamic and reflective Ruby tricks; an experimental branch of JRuby, utilizing the Truffle AST interpreter framework and Graal JIT compilation framework by Oracle Labs, can execute that pure Ruby "fallback version" as fast as the YARV can execute the original highly-optimized C version. This is simply (well, anything but) achieved by some really clever people doing really clever stuff with dynamic runtime optimizations, JIT compilation, and partial evaluation.
Theoretically, if you write code that does exactly the same in two languages and compile both using a "perfect" compiler, the performance of both should be the same.
In practice, there are several reasons why the performance is going to be different:
Some languages are harder to optimize.
This includes especially features that make code more dynamic (dynamic typing, virtual methods, function pointers), but also for example garbage collection.
There are various ways to make code using such features fast, but it usually still ends up at least somewhat slower than without using them.
Some language implementations have to do some compiling at runtime.
This applies especially to languages with virtual machines (like Java) and languages that execute the source code, with no intermediate step for binary (like JavaScript).
Such language implementations have to do more work at runtime, which affects performance, especially startup time/performance soon after startup.
Language implementations intentionally spend less time on optimizations than they could.
All compilers care about performance of the compiler itself, not just of the generated code. This is especially important for runtime compilers (JIT compilers), where taking too long to compile slows down the application execution. But it applies to ahead-of-time compilers, like those for C++ too. For example register allocation is an NP-complete problem, so it's not realistic to solve it perfectly and instead heuristics that execute in reasonable time are used.
Differences in idioms in different languages.
Code written in the style common for a certain language (idiomatic code) using common libraries can result in difference in performance, because such idiomatic code behaves subtly differently in each language.
As an example, consider vector[i] in C++, list[i] in C# and list.get(i) in Java. The C++ code likely does no range checking and performs no virtual calls, the C# code performs range checking, but no virtual calls and the Java code performs range checking and it's a virtual call. All three languages support virtual and non-virtual methods and both C++ and C# can include range checking or avoid it when accessing the underlying array. But the standard libraries of these languages decided to write these equivalent functions differently, and, as a consequence, their performance will be different.
Some compilers might be missing some optimizations.
Compiler writers have finite resources, so they can't implement every possible optimization, even ignoring the other issues. And the resources they have might be focused on one area of optimization more than others. As a consequence, code written in different languages can have different performance because of differences in their compilers, even if there is no fundamental reason why one language should be faster or even easier to optimize than the other.
This is a very general question, but in your case the answer could be simple. C++ compiles to machine code, where Java compiles to Java byte code, which is then run of a Java virtual machine (though there is also just-in-time compiling, which further compiles the Java byte code to native machine code). Another difference could be garbage collection, which is a service that only Java provides.
Simple answer is, the closer a language is to the hardware the faster it would be.
The more a language depends on libraries and other functions to do it's tasks the longer it takes for program execution in that language.
It bears repeating that the algoritm used is most important.
An example might be you writing your oen bubble sort on a million records in the worlds fastest language. Compared it to using a really slow language with a built in really good sort. Your fast language will be mighty slow in comparison.
Your question is too general, so I'm afraid I can't give an exact answer that you need.
For my best explanation, let's look at C++ and .Net platform.
C++, very close to machine code and one of the favor programming in the older time (like more than 10+ years ago?) because of the performance. There isn't much resources needed to develop and execute C++ program even with the IDE, they are consider very light weight and very fast. You can also write C++ code in console and develop a game from there. In terms of memory and resources, I forgotten roughly how much capacity it takes in a computer but the capacity is something that you can't compare with current generation of programming language.
Now let's look at .Net. The prerequisite to do .Net development is one giant IDE that consists of not just one type of programming languages. Even if you just want to developer in let's say C#, the IDE itself will includes many programming platforms by default such as J#, VB, mobile and etc by default. Unless you do custom install and only install exactly what you want, provided that you are experience enough to play with the IDE installation.
Other than installing the IDE software itself, .Net also comes with huge capacity of libraries and frameworks for the purpose of ease of access to any kind of platform that developers need AND don't need.
Developing in .Net can be a fun experience because many common function and components are available by default. You can drag-and-drop and use many validation method, file reading, database access and much more in the IDE.
As a result, it's a trade off between computer resources and speed of development. Those libraries and framework takes up memory and resources. When you execute a program in .Net IDE, it may consume tons of time trying to compile the libraries, components and all of your files. And when you do debugging, your computer require lots of resources to manage the debugging process in your IDE.
Usually, in order to do .Net development, you need a good computer with the some Ram and processor. Otherwise, you might as well don't programming at all. In this aspect, C++ platform is much better than .Net. Although you still need a good computer, but capacity won't be too much of concern compare to .Net.
Hope my answer could help with your question. Let me know in case you want to know more.
EDIT:
Just a clarification to the my main point, I mainly answer to the question of "What's govern the speed of programming".
In IDE standpoint, using C++ or .Net language in the relative IDE does not affect the speed of writing the code, but it will affect the speed of development because Visual Studio compiler take longer time to execute the program, but C++ IDE is much lighter and consume less computer resources. So in the long run, you can do programming faster with C++ type of IDE compare with .Net IDE which heavily depending on libraries and framework. If you take in the amount of time of waiting the IDE to start up, compile, execute the program and etc, it will affect the speed of programming. Unless "the speed of programming" is actually only focus on "speed of writing code".
The amount of libraries and framework in Visual Studio also consume the computer capacity. I'm not sure if this answer the question of "memory management", but I just want to point out that Visual Studio IDE can take up lots of resources when running it, and thus slow down the overall "speed of programming". Of course, this has nothing to do with "speed of writing code".
As you may have guessed, I don't know too much of C++ as I just use it as example, my main point is about Visual Studio type of heavy IDE that consume computer resources.
If I got the idea and did not answer the thread starter question at all, apologize for the long post. But I would advice the thread starter to make the question clear and ask exactly what he/she need to know about "faster" and "slower"
