^+^+^+^+^+^+^+^+^+^+^+^+^+^+^+^+^+^+^+^+^+^+^+^+^+^+^+ EEMBC JOURNAL ^+^+^+^+^+^+^+^+^+^+^+^+^+^+^+^+^+^+^+^+^+^+^+^+^+^+^+ NEWS FROM EMBEDDED MICROPROCESSOR BENCHMARK CONSORTIUM ^+^+^+^+^+^+^+^+^+^+^+^+^+^+^+^+^+^+^+^+^+^+^+^+^+^+^+ Autumn 2003 ^+^+^+^+^+^+^+^+^+^+^+^+^+^+^+^+^+^+^+^+^+^+^+^+^+^+^+ In this issue 1. From Complexity Comes Growth - Letter from the President 2. What's in the new benchmark suites? 3. Q3 Benchmark Score Reports 4. Upcoming Events 5. Publication News 6. EEMBC Welcomes DCT Ltd. 7. From the Lab _________________ 1. From Complexity Comes Growth - Letter from the President The first-generation EEMBC benchmarks, released in 1998, were designed with flexibility in mind - and to run on devices from high-end processors to 8-bit microcontrollers. Just five years ago, it was not uncommon to find processor evaluation boards with less than 1M of system memory, so many of these benchmarks were crafted to focus on the behavior of the processor core and the compiler. This focus remains significant for processor vendors and system designers, but now the industry is requesting more complex benchmarks that stress the processor and its ability to interface with the entire memory subsystem. The first-generation EEMBC benchmarks, released in 1998, were designed with flexibility in mind - and to run on devices from high-end processors to 8-bit microcontrollers. Just five years ago, it was not uncommon to find processor evaluation boards with less than 1M of system memory, so many of these benchmarks were crafted to focus on the behavior of the processor core and the compiler. This focus remains significant for processor vendors and system designers, but now the industry is requesting more complex benchmarks that stress the processor and its ability to interface with the entire memory subsystem. EEMBC's new Digital Entertainment and Networking v2.0 benchmarks (see accompanying table and From the Labs column by Alan Weiss on page 3), are a huge step in the direction of greater complexity while maintaining the original principles underlying the Consortium. Take the new MPEG-4 benchmark, for example. Many processor vendors already have various metrics to demonstrate the capability of their processors performing MPEG-4 decoding, but these are not really designed to facilitate comparisons among devices. By contrast, the MPEG-4 decode EEMBC benchmark uses a standard code base and input data files. This ensures that all vendors are running the same MPEG-4 implementation to allow an apples-to-apples comparison. Another hallmark of the EEMBC process has been our insistence on score certification as a condition of publication. Now this process is becoming more important than ever. With such complex benchmarks, the potential for errors-and the temptation to "cheat"-grows exponentially. We've designed the Digital Entertainment and Networking v2.0 benchmarks, and their certification protocols, with this thought in mind, and the new suites will be highly resistant to manipulation. With these and other benchmarks under development, we'repositioning EEMBC to build on the successes of the past five years by addressing new embedded applications on the horizon within the framework of fairness and objectivity that has made EEMBC scores an industry standard. Interest in the Consortium, among both processor and compiler suppliers and their customers, has reached a new level. But we see more clearly now that we're just beginning. As our benchmarks become still more sophisticated and able to show processor and compiler capabilities in sharper detail, we're looking forward to another surge in membership and further reliance on certified EEMBC scores by scores of system designers. Markus Levy EEMBC President _________________ 2. What's in the new suites? Details of the new Digital Entertainment and Networking v2.0 benchmarks are listed below. Digital Entertainment MP3 Decode - Addresses implementations of MP3 decode in hardware with a focus on reprogrammable solutions for mobile phones, PDAs, and similar devices. MPEG-4 Video Codec - Focused on the mobile market, new benchmarks include separate encode and decode kernels. MPEG-2 Video Codec - Focused on higher-end products such as set top boxes, with separate encode and decode kernels. Cryptography - Requiring very data intensive processing, addresses rapidly rising demand for random number generation to facilitate financial transactions across the Internet. Networking V2.0 OSPF (Open Shortest Path First)/Dijkstra: Implements the Dijkstra shortest path first algorithm, which is widely used in routers and other networking equipment. Route Lookup/Patricia: A distillation of the fundamental operation of IP datagram routers: receiving and forwarding IP datagrams. QoS (Quality of Service): A guarantee of transmission rates, error rates, and other characteristics to improve on the "best effort" protocols of the public network. QoS requires that packets passing through a gateway host can be expedited based on policy and reservation criteria arranged in advance. IP Packet Check: Emulates how IP datagrams are often stored in actual systems using descriptors which are separate from the datagram. IP Reassembly: Simulates the processing that handles reassembly when large packets are split up. NAT (Network Address Translation): The translation of an IP address used within one network to a different IP address known within another network. TCP: A protocol used along with the IP to send data in the form of message units between computers over the Internet. TCP splits the packets of a file, numbers the packets, and then forwards the individual packets to the IP program layer. _________________ 3. Q3 Benchmark Score Reports AMD Alchemy and PMC-Sierra published new benchmark scores in the third quarter of 2003, bringing the total number of score reports available for free on the EEMBC web site to more than 260. Score details: AMD Alchemy: out-of-the-box scores on the 396-MHz Au1100 in all five EEMBC application-based benchmark suites. PMC-Sierra: out-of-the-box scores on the 625-MHz RM7000C in all five EEMBC application-based benchmark suites. Complete details are available from the Search Benchmark Scores page < http://www.eembc.org/benchmark/benchmain.asp> on the EEMBC Web site. _________________ 4. Upcoming Events October 14 - Visit EEMBC at the Microprocessor Forum Expo at the San Jose Fairmont Hotel, 5:30 p.m. to 8:30 p.m. The industry's premier event for first public disclosures of new processor architectures, this year's MPF (being held October 13-16) will include presentations from EEMBC members ARC International, ARM, Fujitsu, IBM, Imagination Technologies, MIPS Technologies, Motorola, Philips, Renesas, Sun Microsystems, and SuperH. For further information about Microprocessor Forum, visit http://www.mdronline.com/mpf/. October 30 - "Platform-Based Design Trends," a 90-minute program sponsored by the The San Diego Telecom Semiconductor SIG, will be moderated by EEMBC President Markus Levy. He'll be joined by Chris Ward (Director of Technical Sales at ARM), Mazen Chmaytelli (Senior Product Manager for QUALCOMM Internet Services), and Drew Wingard (CTO at Sonics, Inc.) to explore what platform-based design means, how it relates to mobile convergence, and why open, standardized interfaces are becoming so important. The meeting runs 7:30 a.m. to 9:30 a.m. Further information is available from http://www.sdtelecom.org. _________________ 5. Publication News EEMBC has been invited to contribute on a regular basis to Embedded Computing Design Magazine, beginning in the Fall 2003 issue with the overview article "EEMBC's performance benchmarks for embedded processors, compilers, and Java platforms" by Markus Levy. Visit the magazine at (http://www.embedded-computing.com). For a more in depth analysis of the EEMBC Java benchmarks, see "Grinding Up Embedded Java Platforms with Industry-Standard Benchmarks" by David Proulx of Sun Microsystems and Chair Pro-Tem of the EEMBC Java Subcommittee, coming up in the October issue of ECE Embedded Control Europe (http://www.embedded-control-europe.com/magazine). In the next (Volume 2, Number 3) issue of IQ: The Information Quarterly, Markus Levy shows how "Processor Performance Drives In-Car Entertainment" (http://www.convergencepromotions.com/IQ), while EEMBC's new Digital Entertainment benchmarks get an in-depth treatment in Levy's "Benchmarking Processors for Multimedia Applications," scheduled for publication later this month in Electronics Weekly (http://www.electronicsweekly.co.uk/). _________________ 6. EEMBC Welcomes DCT Ltd. Digital Communication Technologies (DCT) is the latest player to join EEMBC's Java subcommittee, which has developed the industry's first objective, certifiable, and real-world benchmarks for Java J2ME implementations. In business since 2000, DCT recently launched its BigfootT Java acceleration technology, which combines hardware and software to deliver faster load times and more efficient use of static and dynamic memory in Java-enabled wireless handsets. "DCT is pleased to join EEMBC and looks forward to actively participating in and supporting the efforts of the Java Subcommittee," said Scott Avery, vice president of marketing and business development for DCT. "Since Java has become the standard for delivering application content for mobile phones, customers are insisting on utilizing an objective, application-specific benchmark to determine which Java acceleration technique best meets their design criteria. We are confident that the EEMBCR Java benchmarks will further demonstrate the performance advantages of our Bigfoot acceleration technology." Visit DCT on the Web at http://www.dctl.com. _________________ 7. From the Lab Networking subcommittee, took the lead on developing the Networking v2.0 code, which ECL turned into EEMBC standard benchmarks. The MPEG-2 code is built out of the MPEG Committee's reference source code, and includes both encode and decode benchmarks across a variety of bitstream s. The data sets were chosen to be challenging, and initial tests show they are succeeding. While "Marsface" (a black and white image of the Face on Mars at Cydonia rotating) isn't very stressful, the "graphic.mpeg" file at 720x480, 30 frames a second, at 50 frames (for decoding) or 7 frames (for encoding) seems to be really taxing on systems. By comparison, the MP3 data sets (Holsts' Jupiter, and a test reference file) are much easier to process (and MP3 is a decode-only benchmark), but the planned Quality Metric for both should give readers of the benchmark scores a good idea about performance - and catch cheaters, too. The MPEG-4 code came from the open source movement and a group called XviD. ECL worked closely with the XviD community, so much so that our bug fixes were incorporated into their standard source code releases. The most difficult technical challenge was turning code that expected the presence of a filesystem into truly embedded code. ECL developed a program called "cheader" (alternatively pronounced "See-header" or "cheat-er") that encapsulates any arbitrary dataset into a C language header file, then builds that as a separate object to be linked in by the linkage editor. Because this is in a library, multiple benchmarks can share files out of the same media library, giving independence between datasets and the actual benchmark code itself. The filesystem C code semantics (POSIX stream I/O) are preserved with the same performance as static arrays, and eliminates code changes required around the file code logic. Thus, an embedded system uses memory instead of a disk for the datasets in a clever, but straightforward manner. This abstraction is a key feature of EEMBC's approach to benchmarking, and ECL plans on using this technique for the upcoming Office Automation Version 2 benchmarks. The dramatically expanded Networking Version 2 suite includes a completely revised and enhanced Packet Check (now called IP_Packetcheck), IP_Reassembly, and Open Shortest Path Function (OSPF Version 2). In addition, there are totally new benchmarks covering Quality of Service (QoS) and Network Address Translation (NAT). ECL was particularly concerned about the initial state of the QoS code, since it used many unions and bit-masking to determine which packets were to be processed ahead of others. As it turns out, our solution to the non-portability concern was to realize that the initialization code was expecting that all malloc() calls were returning zero-initialized storage, and some of the low-level pointers in the data structures were not specifically initialized. We implemented a new call in the Test Harness, th_calloc() to resolve this. As a byproduct, this became an aggressive C compiler test, and it certainly tests how well processors can manipulate and process packets. The Networking Version 2 benchmarks also contains brand new TCP code, provided by Motorola and implementing both a client and a server. Sun and Markus Levy worked with network equipment provider Juniper Networks to garner requirements for these benchmarks. The result will be two new "Marks": TCPmark and IPmark.. From a logistics perspective, Networking Version 2, unlike Consumer, will replace Version 1.1, as it is a superset. The datasets are now parameter driven, and the amount of work is significantly expanded to meet a key requirement of stressing L2 and even L3 caches, rather than just fitting into small L1 caches. Given these exciting, sophisticated new benchmarks, we believe OEM's will increasingly demand certified EEMBC benchmark scores before selecting processors for their designs. Alan R. Weiss Chairman and CEO, EEMBC Certification Labs (ECL) alan@ebenchmarks.com _________________ If you do not wish to receive e-mail from EEMBC, you can un-subscribe by accessing the following link: http://www.eembc.org/asp/unsubscribe.asp. EEMBC sends no more than one e-mail per month to registered users at www.eembc.org. Continuing your subscription ensures you'll be notified when new scores and other important announcements are available.