Into the Fast Stream: Real-time Web Monitoring with StreamWeb

Web services such as Twitter work by using streaming Web APIs (Application Programming Interfaces) – a set of codes that software programs follow in order to communicate with each other. APIs enable users to send and retrieve streaming data, meaning it is now possible to follow the current status or trends of the world in real-time, such as tracking of infectious disease, crime prediction, or marketing. Various innovative business services are therefore possible if streaming can be monitored successfully. However, for such services to be implemented into a system at present, the performance and scalability of the services will depend on the skills of the computer engineers.

To make this task easier, Toyotaro Suzumura and Tomoaki Oiki based in the Department of Computer Science at Tokyo Institute of Technology, have created a real-time web monitoring system called ‘StreamWeb’ that works alongside a stream computing system called System S developed by IBM Research. The StreamWeb system allows developers to easily input analytical algorithms for a variety of different streaming data without worrying about performance and scalability. StreamWeb also provides real-time web monitoring for massive amounts of data.

The researchers created an application that monitors a list of keywords in Twitter streaming data. The Twitter messages are then displayed, with the specified keywords highlighted, on a Google map showing the physical location where the message was posted. The application can handle nearly 30 thousand Twitter messages per second on a system with 8 computing nodes. This prototype application confirms that web monitoring systems can be built that satisfy the need for high software productivity and can also be used on systems of different scales.

• Authors: Toyotaro Suzumura and Tomoaki Oiki
• Title of original paper: StreamWeb: Real-Time Web Monitoring with Stream Computing
• Journal, volume, pages and year: IEEE International Conference on Web Services, ICWS 2011, 620-627, 2011
• Digital Object Identifier (DOI): 10.1109/ICWS.2011.16
• Affiliations: Department of Computer Science, Engineering, Graduate School of Information Science and Engineering, Tokyo Institute of Technology
• Department website: http://www.cs.titech.ac.jp/

The apparatus for cold electron collision experiments employing the threshold photoelectron source.