EEMBC JOURNAL NEWS FROM THE EMBEDDED MICROPROCESSOR BENCHMARK CONSORTIUM www.eembc.org Summer 2003 _________________ In this issue 1. Raising the Bar - Letter from the President 2. From the Lab 3. Q2 Benchmark Score Reports 4. New Literature Library CD ROM Collects Data Sheets, Score Reports, and More 5. Apologies to NEC's Mr. Lamar _________________ 1. Raising the Bar - Letter from the President In the past year, we've seen a threefold increase in the number of EEMBC members who are certifying and publishing benchmark scores for their embedded microprocessors. Members like IBM, Motorola, NEC Electronics, SuperH, and Tensilica are actively using EEMBC benchmark scores in their product advertising and other promotions. The word on the street is that more OEMs are asking for EEMBC scores and more often. Even more impressive is the increasing "behind the scenes" use of EEMBC benchmarks at member companies to analyze, tune, and validate new processor architectures. This trend was much in evidence at the Embedded Processor Forum (EPF) in San Jose last month. ARM, MIPS Technologies, SuperH and Texas Instruments were among the EPF presenters who mentioned having used EEMBC benchmarks to either demonstrate performance or guide architectural decisions. For example, ARM used EEMBC to help prove that its full ARM mode was only a few percent faster than its new Thumb-2 mode, even though code size for Thumb-2 is significantly smaller. MIPS Technologies pointed out how it used EEMBC to help design the dynamic branch predictor of its new MIPS 24K microarchitecture. From the beginning, EEMBC and its members have earnestly taken great care to ensure that its benchmarks are representative of real-world applications. Now this is paying off. Although the best benchmark will always be the customer's application, EEMBC's benchmarks have become the next-best thing. The shear breadth of the benchmark suites is a remarkable achievement, one that gives members a practical way to raise the performance bar for their processors and compilers. And there is no doubt that this bar has been raised significantly over the last few years. The situation is about to get even better. New EEMBC media and networking benchmarks to be launched this year will combine with the original EEMBC benchmarks to boost our code base to well over 1 million lines of code. To top it off, all these new kernels fit within a Portable Test Harness framework that makes it easier than ever to run the benchmarks (see "From the Labs" on page ##). We expect that members will be eager to publish scores based on these new benchmarks, which will measure device and compiler performance in such tasks as MPEG-2, MPEG-4, and MP3 conversion, TCP/IP, QOS, network address translation, and more. Beyond this, we'll be providing a tool that will help suppliers at a much earlier stage to develop architectures that handle these real-world tasks faster. And this points to the larger significance of what EEMBC is doing. It's not just about measuring the performance of existing processors. Our bigger contribution to the industry is in helping to make future compilers and processors better. Markus Levy EEMBC President _________________ 2. From the Lab Before EEMBC benchmarks are run on an embedded processor, the Test Harness must be ported to the device and its associated platform. It is therefore difficult to underestimate the importance of a recent reorganization and set of improvements to our Portable Test Harness for EEMBC's Version 1.1 and subsequent generations of benchmarks. Based on feedback from a number of dedicated EEMBC members, working with the Test Harness has become--well, if not a pleasure, then at least relatively easy. The development process has included ECL's creation of a "Test Harness Lite," an implementation of self-checking on both the regular and Lite versions, and, most recently, creation of a "Test Harness Ultra Lite" for the 8/16-Bit Microcontroller Benchmark Subcommittee. The Test Harness (TH) is one of the key elements of EEMBC's success because it provides a number of useful functions: Abstraction. TH abstracts the individual benchmarks from the hardware, so the benchmark engineer does not have to port each benchmark - only the platform-dependent portion of the Test Harness. A change in one particular benchmark does not require a recompilation, or change of code, in the rest of the benchmarks or the Harness itself. Portability. By locating the platform-dependent portion of the code into the Adaptation Layer, the benchmarks can be easily ported between processors in a family or between processor architectures. Automation. With the new TH in EEMBC 1.1 and forthcoming "Version 2" benchmarks, the new makefile system automates the build, run, and gathering data steps for the benchmark process. Flexibility. Moving to gcc and makefiles as the default build method has more closely aligned EEMBC code with our members' development tools. EEMBC also supports any IDE- and GUI-driven toolset. In fact, the statistics indicate that the most frequently used tool chain used by EEMBC members for certifications is Green Hills Multi, an IDE-based tool. Top-down and local builds are easily accomplished, and the standardized naming and directory structure simplifies compliance. The Test Harness for benchmarks now under development adds a number of new features: a RAMfile system for all of the streams calls a new function in the platform independent section for randomization th_rand() a printf() and uuencode() function for Test Harness Lite closer matching of eembc_dt.h data type abstraction files between TH Regular and TH Lite support for very large benchmarks The Test Harness system helps to simplify the use of EEMBC benchmarks. It therefore plays a key role in supporting the use of the benchmarks in our members' R&D, marketing, and sales activities while making it possible for processor specifiers to make straightforward comparisons of processors, tools (compilers), libraries, and related technologies. Alan R. Weiss ECL Chairman and CTO alan@ebenchmarks.com _________________ 3. Q2 Benchmark Score Reports IBM, Intrinsity, SuperH, and Texas Instruments published new benchmark scores in the second quarter of 2003, bringing the total number of score reports available for free on the EEMBC web site to more than 250. Score details: IBM: out-of-the-box scores on the 667-MHz PowerPC 440GX in all five EEMBC application-based benchmark suites. Intrinsity: out-of-the-box and assembly-optimized telecom scores on the 2-GHz FastMATH processor SuperH: simulated assembly-optimized consumer and automotive/industrial scores on the 266-MHz SH4-202. Texas Instruments: out-of-the-box, C-optimized, and assembly-optimized telecom scores on the 720-MHz TMS320C6414. Complete details are available from the Search Benchmark Scores page on the EEMBC Web site. _________________ 4. New Literature Library CD ROM Collects Data Sheets, Score Reports, and More A new EEMBC CD-ROM launched at the June Embedded Processor Forum includes EEMBC benchmark data sheets, white papers, and all benchmark score reports published as of Q2 2003. Score reports are offered in both pdf and Excel form. The disc also includes useful overviews on the Consortium's activities and information on EEMBC Certification Labs (ECL) services. To obtain a copy, please contact Rae Morrow, Wall Street Communications at rae.morrow@wallstcom.com. _________________ 5. Apologies to NEC's Mr. Lamar Due to a copyediting error, a story on EEMBC subcommittee chairs in the Spring 2003 issue of this newsletter omitted the name of David Lamar, the dedicated and longstanding chair of the Consortium's 8/16-Bit Microcontroller Subcommittee. We sincerely regret this inexplicable oversight. _________________ If you do not wish to receive e-mail from EEMBC, you can un-subscribe by accessing the following link: http://www.eembc.org/asp/unsubscribe.asp. EEMBC sends no more than one e-mail per month to registered users at www.eembc.org. Continuing your subscription ensures you'll be notified when new scores and other important announcemen