|By Java News Desk||
|June 18, 2004 12:00 AM EDT||
On Monday we published an article about Keith Lea's one-man attempt to take the benchmark code for C++ and Java from the now outdated Great Computer Language Shootout conducted by Doug Bagley in Fall 2001 and run the tests himself. Within 12 hours 40,000 people had read the piece and by the end of the day Bagley's "Shootout" had even been revived as an ongoing Debian Alioth project.
But while the team behind that project cheerfully says on its Web site "Contributions are welcomed for revised implementations, bug fixes, or ideas for new benchmarks," not everyone agrees with this kind of activity.
When Bagley did the original shootout, he said it was "To learn and to have fun," adding that he would continue continue "as long as the fun holds out." This notion of enjoyment has been revived by the Debian Alioth update:
"The project goals have not changed substantially since Doug's original project. This work is continuing so that we all can learn about new languages, compare them in various (possibly meaningless) ways, and most importantly, have some fun!"But many of those who troubled to record their views in the course of the past few days did not recognize benchmarking/microbenchmarking as anything worth doing at all.
Some took issue with Lea's choice of performance speed as the parameter to compare Java and C++. One, Lachlan Stuart, wrote:
"What you must realize is that C++ and Java have much different targets, it's rare that you will encounter a situation where both are equally suitable. No matter what, unless you forget to turn debug compile off, C++ will beat Java in real world situations on the performance front..."
"Try praising something else about Java...like its security, or ease of programming," Stuart continued.
GooberToo concurred. "The more I read the code, it sure is starting to look more and more like an apples and oranges comparison; which is usually what happens when people do Java benchmarks." And Iain too was unhappy: "Aside from perhaps demonstrating that you know Java far better than you know C++, I don't think these tests were a particularly useful exercise."
Others too felt that Lea's relative unfamiliarity with C++ compromised the exercise. "This article is completely useless and only shows the lack of understanding of C++ by the author," wrote Dr Valentin Samko. "This reminds me of a person who clamed that qbasic is faster than C++ because his program which outputs a few million blank lines works faster in qbasic than in C++."
Glen Low queried the shootout's methodology on a number of counts. Here is one:
"The small size of the programs are a definite advantage for Java's runtime optimizer. A run-time optimizer can't spend forever optimizing, but it has access to actual run-time data, AND can specialize its optimization for the specific processor. (G++, and most other C++ compilers, have flags to allow this as well.) Given the small size, a complete analysis of the program is possible in the limited time available, and given the other additional data, I would find it rather worrisome if Java didn't do better. I'm more sceptical with regards to real programs, however. If there really is one small critical loop, Java should win; that's not the case for most of my programs, however."Scott Ellsworth disagreed with those who dismissed Lea's shootout, though. "The point is that many C++ users feel that Java is too slow and uses too much memory. Microbenchmarks like this show that this off the cuff response is as silly as claiming that exceptions cost too much just because cfront compilers had that cost," he wrote.
"I spent seven years doing C++ in a shrink wrap environment, and five doing Java. In my experience with real world apps, Java code gets done faster, works better out of the box, and is easier to maintain. Tricks like custom memory allocators and carefully crafted templates can make faster C++ code, but I reserve those for time critical parts of the program. Where I do not spend some programmer cycles profiling and optimizing, the two environments are pretty close.Patrik Beno issued a challenge. "I suggest that now it's time for C++ experts to optimize C++ code and compilation, run the very same tests again and post the results."
Thus - if Java is producing code reasonably close or ahead in fairly straightforward usage, and the code gets written faster and more portably, then Java is a win for my work. I can always choose to use C++, assembly, or FORTRAN for the time critical routines, and then link in."
"Don't blame Lea, he did his best, now it's your turn, C++ experts," he added.
Perhaps SWagner would be someone who will rise to the challenge in due course. S/he wasn't at all happy with Lea, writing:
"As an experienced C++ developer (and Java as well) I only see that the C++ code is optimized for minimum performance. Especially the methodcall where the class has virtual constructors or destructors which is seldom useful and significantly hurts performance. Just because you can't turn this off in Java does not mean that you have to make it slow in C++.""In my practical experience," continued Wagner, "normal Java code is 1000 times slower [compared] to OPTIMIZED C++. However if you make severe design flaws you can make any program slow. And if you base on a bad benchmark and make it even worse you should better stop any benchmarking you planned for the future."
Somewhat less disgruntled was m, who wrote: "The benchmarks are interesting, but frankly with real applications, I have never experienced Java performance equal to or better than that of C++."
"I have done a number of home-grown benchmarks for XML parsing, and C/C++ (think expat) are 3-9 times faster," m continued. "The difference there is that the server JVM does indeed optimize over a series of many runs. Java certainly allows us to develop faster, and quite often time to market performance is more important than the performance differences in these two (great in my opinion) platforms. They each have specific advantages."
This, not surprisingly, was the essence of quite a few of the comments. "Java is faster for some applications than C++, and for others C++ is faster. It depends on what you want to do with it," said RebelCool.
But Iain's objection was to the choice of tools to compare, to the entire premise of the shootout in other words:
"The comparison [Lea is] trying to make between C++ and Java is not particularly meaningful. At the end of the day, if a VM is faster, then there is nothing to stop you implementing a VM in C++ and running the 'C++ program' within that framework. What times would then be valid? The tools you are comparing are very different.
In addition, the C++ code is seriously sub-optimal. To take one example, the 'matrix.cpp' is definitely not written for speed. A general purpose C function is being used to repeatedly allocate rows, when a single large block of size 'mn' could have been used. For that matter, why is malloc( ) being used at all? (A custom memory manager optimised for small blocks could have been used instead - malloc( ) is slow.)
If you insist on writing simple C, then at least use memset( ) for the zeromatrix( ) function. Fast template matrix libraries exist for C++ - these could have been used instead - the result would have been shorter, clearer and considerably faster. Aside from perhaps demonstrating that you know Java far better than you know C++, I don't think these tests were a particularly useful exercise."
NotZed disagreed with the premise too, writing: "Java works well as a COBOL replacement, a backend application language for boring buisiness apps. That's what it should be benchmarked against. Comparing it to C is like comparing apples and oranges."
Chris countered: "Surely the benchmark is no less meaningful than when it was originally used to show C++ was faster than Java."
And Christopher White commented:
"Thanks for the comparison Keith. I enjoyed reading your commentary.
Many people have stated in response to this article that Java, as everyone knows, is slow... This is certainly not the case. I am team leader of a Java development team. We have six applications in production on a certain open source application server. We have 1200 users of these applications. The applications are load-balanced over two machines (each machine : HP DL380 2*2.8GHz 2Gb RAM). We have on average 2% Cpu utilisation. Whilst this is of course just a subjective view of Java performance, I can say that we certainly do not have the impression that Java is at all slow.
I accept that for memory deallocation C++ can be tailored to the needs of the application at hand so it will be difficult to beat on this score, however one must remember that the just in time compiler compiles code into native machine language and that this compiler is very much optimized on a per platform basis so I don't see why C++ should produce naturally faster compiled code.
Whichever is faster, Java is easily fast enough for most applications and I strongly believe that the overhead in development time incurred by using C++ is rarely justified except in specialized applications such as driver or game coding."
For the moment let's give the last word to SB:
"Java and C++ are different languages, with different advantages/drawbacks, and different philosophies. My experience, from a large company that produces tons of software, is that performance always comes down to the quality of the developers, not the language itself. And we also found out that in today's world, most developers are only average, and this class of developers yields better results with Java than C++."The debate, as they say, will doubtless continue...
|icube 06/29/04 05:47:18 AM EDT|
shootout has a great interest but not between C++ and java but between ocaml and java .... :-))))
be aware & wake up !!!
|microbenchmarks 06/21/04 06:47:50 AM EDT|
Doug Bagley''s original Great Computer Language Shootout was a neat idea: a collection of little benchmarks written in several dozen languages for the purposes of comparing speed, memory usage, and lines of code. He used to say that his favorites of all the languages were those in the ML family, and in particular Ocaml.
|Robert McGovern 06/20/04 04:33:36 PM EDT|
comp.lang.c++.moderated list at Google Groups. At least be accurate and say on the newsgroup comp.lang.c++.moderated, rather than push the notion that the group is only available via Google groups.
According to Forrester Research, every business will become either a digital predator or digital prey by 2020. To avoid demise, organizations must rapidly create new sources of value in their end-to-end customer experiences. True digital predators also must break down information and process silos and extend digital transformation initiatives to empower employees with the digital resources needed to win, serve, and retain customers.
Oct. 28, 2016 02:45 AM EDT Reads: 1,780
SYS-CON Events announced today that SoftNet Solutions will exhibit at the 19th International Cloud Expo, which will take place on November 1–3, 2016, at the Santa Clara Convention Center in Santa Clara, CA. SoftNet Solutions specializes in Enterprise Solutions for Hadoop and Big Data. It offers customers the most open, robust, and value-conscious portfolio of solutions, services, and tools for the shortest route to success with Big Data. The unique differentiator is the ability to architect and ...
Oct. 28, 2016 02:30 AM EDT Reads: 1,177
You think you know what’s in your data. But do you? Most organizations are now aware of the business intelligence represented by their data. Data science stands to take this to a level you never thought of – literally. The techniques of data science, when used with the capabilities of Big Data technologies, can make connections you had not yet imagined, helping you discover new insights and ask new questions of your data. In his session at @ThingsExpo, Sarbjit Sarkaria, data science team lead ...
Oct. 28, 2016 02:00 AM EDT Reads: 4,385
A completely new computing platform is on the horizon. They’re called Microservers by some, ARM Servers by others, and sometimes even ARM-based Servers. No matter what you call them, Microservers will have a huge impact on the data center and on server computing in general. Although few people are familiar with Microservers today, their impact will be felt very soon. This is a new category of computing platform that is available today and is predicted to have triple-digit growth rates for some ...
Oct. 28, 2016 02:00 AM EDT Reads: 34,365
In past @ThingsExpo presentations, Joseph di Paolantonio has explored how various Internet of Things (IoT) and data management and analytics (DMA) solution spaces will come together as sensor analytics ecosystems. This year, in his session at @ThingsExpo, Joseph di Paolantonio from DataArchon, will be adding the numerous Transportation areas, from autonomous vehicles to “Uber for containers.” While IoT data in any one area of Transportation will have a huge impact in that area, combining sensor...
Oct. 28, 2016 01:00 AM EDT Reads: 1,209
SYS-CON Events announced today that Transparent Cloud Computing (T-Cloud) Consortium will exhibit at the 19th International Cloud Expo®, which will take place on November 1–3, 2016, at the Santa Clara Convention Center in Santa Clara, CA. The Transparent Cloud Computing Consortium (T-Cloud Consortium) will conduct research activities into changes in the computing model as a result of collaboration between "device" and "cloud" and the creation of new value and markets through organic data proces...
Oct. 28, 2016 12:45 AM EDT Reads: 1,519
SYS-CON Events announced today that Niagara Networks will exhibit at the 19th International Cloud Expo, which will take place on November 1–3, 2016, at the Santa Clara Convention Center in Santa Clara, CA. Niagara Networks offers the highest port-density systems, and the most complete Next-Generation Network Visibility systems including Network Packet Brokers, Bypass Switches, and Network TAPs.
Oct. 28, 2016 12:00 AM EDT Reads: 1,454
In today's uber-connected, consumer-centric, cloud-enabled, insights-driven, multi-device, global world, the focus of solutions has shifted from the product that is sold to the person who is buying the product or service. Enterprises have rebranded their business around the consumers of their products. The buyer is the person and the focus is not on the offering. The person is connected through multiple devices, wearables, at home, on the road, and in multiple locations, sometimes simultaneously...
Oct. 27, 2016 10:30 PM EDT Reads: 3,843
Everyone knows that truly innovative companies learn as they go along, pushing boundaries in response to market changes and demands. What's more of a mystery is how to balance innovation on a fresh platform built from scratch with the legacy tech stack, product suite and customers that continue to serve as the business' foundation. In his General Session at 19th Cloud Expo, Michael Chambliss, Head of Engineering at ReadyTalk, will discuss why and how ReadyTalk diverted from healthy revenue an...
Oct. 27, 2016 10:30 PM EDT Reads: 3,088
SYS-CON Media announced today that @WebRTCSummit Blog, the largest WebRTC resource in the world, has been launched. @WebRTCSummit Blog offers top articles, news stories, and blog posts from the world's well-known experts and guarantees better exposure for its authors than any other publication. @WebRTCSummit Blog can be bookmarked ▸ Here @WebRTCSummit conference site can be bookmarked ▸ Here
Oct. 27, 2016 10:30 PM EDT Reads: 9,810
Intelligent machines are here. Robots, self-driving cars, drones, bots and many IoT devices are becoming smarter with Machine Learning. In her session at @ThingsExpo, Sudha Jamthe, CEO of IoTDisruptions.com, will discuss the next wave of business disruption at the junction of IoT and AI, impacting many industries and set to change our lives, work and world as we know it.
Oct. 27, 2016 10:30 PM EDT Reads: 565
Established in 1998, Calsoft is a leading software product engineering Services Company specializing in Storage, Networking, Virtualization and Cloud business verticals. Calsoft provides End-to-End Product Development, Quality Assurance Sustenance, Solution Engineering and Professional Services expertise to assist customers in achieving their product development and business goals. The company's deep domain knowledge of Storage, Virtualization, Networking and Cloud verticals helps in delivering ...
Oct. 27, 2016 10:15 PM EDT Reads: 1,166
The Internet of Things (IoT), in all its myriad manifestations, has great potential. Much of that potential comes from the evolving data management and analytic (DMA) technologies and processes that allow us to gain insight from all of the IoT data that can be generated and gathered. This potential may never be met as those data sets are tied to specific industry verticals and single markets, with no clear way to use IoT data and sensor analytics to fulfill the hype being given the IoT today.
Oct. 27, 2016 10:00 PM EDT Reads: 2,916
SYS-CON Events announced today that Hitrons Solutions will exhibit at the 19th International Cloud Expo, which will take place on November 1–3, 2016, at the Santa Clara Convention Center in Santa Clara, CA. Hitrons Solutions Inc. is distributor in the North American market for unique products and services of small and medium-size businesses, including cloud services and solutions, SEO marketing platforms, and mobile applications.
Oct. 27, 2016 10:00 PM EDT Reads: 2,124
OnProcess Technology has announced it will be a featured speaker at @ThingsExpo, taking place November 1 - 3, 2016, in Santa Clara, California. Dan Gettens, OnProcess’ Chief Analytics Officer, will discuss how Internet of Things (IoT) data can be leveraged to predict product failures, improve uptime and slash costly inventory stock. @ThingsExpo is an annual gathering of IoT and cloud developers, practitioners and thought-leaders who exchange ideas and insights on topics ranging from Big Data in...
Oct. 27, 2016 09:45 PM EDT Reads: 560
SYS-CON Events announced today that Enzu will exhibit at the 19th International Cloud Expo, which will take place on November 1–3, 2016, at the Santa Clara Convention Center in Santa Clara, CA. Enzu’s mission is to be the leading provider of enterprise cloud solutions worldwide. Enzu enables online businesses to use its IT infrastructure to their competitive advantage. By offering a suite of proven hosting and management services, Enzu wants companies to focus on the core of their online busine...
Oct. 27, 2016 07:45 PM EDT Reads: 1,467
The explosion of new web/cloud/IoT-based applications and the data they generate are transforming our world right before our eyes. In this rush to adopt these new technologies, organizations are often ignoring fundamental questions concerning who owns the data and failing to ask for permission to conduct invasive surveillance of their customers. Organizations that are not transparent about how their systems gather data telemetry without offering shared data ownership risk product rejection, regu...
Oct. 27, 2016 07:15 PM EDT Reads: 329
The Open Connectivity Foundation (OCF), sponsor of the IoTivity open source project, and AllSeen Alliance, which provides the AllJoyn® open source IoT framework, today announced that the two organizations’ boards have approved a merger under the OCF name and bylaws. This merger will advance interoperability between connected devices from both groups, enabling the full operating potential of IoT and representing a significant step towards a connected ecosystem.
Oct. 27, 2016 06:15 PM EDT Reads: 1,335
November 1–3, 2016, at the Santa Clara Convention Center in Santa Clara, CA. Penta Security is a leading vendor for data security solutions, including its encryption solution, D’Amo. By using FPE technology, D’Amo allows for the implementation of encryption technology to sensitive data fields without modification to schema in the database environment. With businesses having their data become increasingly more complicated in their mission-critical applications (such as ERP, CRM, HRM), continued ...
Oct. 27, 2016 05:45 PM EDT Reads: 1,197
SYS-CON Events announced today that Embotics, the cloud automation company, will exhibit at the 19th International Cloud Expo, which will take place on November 1–3, 2016, at the Santa Clara Convention Center in Santa Clara, CA. Embotics is the cloud automation company for IT organizations and service providers that need to improve provisioning or enable self-service capabilities. With a relentless focus on delivering a premier user experience and unmatched customer support, Embotics is the fas...
Oct. 27, 2016 05:00 PM EDT Reads: 1,090