K42

About this project

The K42 Project

Our Mission The K42 group is developing a new high performance, open source, general-purpose research operating system kernel for cache-coherent multiprocessors. We are targeting next generation servers ranging from small-scale multiprocessors that we expect will become ubiquitous, to very large-scale non-symmetric multiprocessors that are becoming increasingly important in both commercial and technical environments. By designing the system from the start for multiprocessors, we achieve a high degree of spatial and temporal locality in code and data. This locality results in substantial performance advantages, even on small-scale servers. To support large servers, K42 is designed to scale to hundreds of processors and to address the reliability, fault-containment, and fault-tolerance requirements of large commercial applications. K42 employs building-block technology to allow applications to customize and thus optimize the OS services they require. This is particularly important for applications, such as databases and web servers, where given the ability to control physical resources, they can improve performance. K42 is portable; it currently runs on PowerPC systems (both native and using Mambo). K42's design allows implementors on a particular architecture to choose what objects of the system should be customized for that architecture, and as a result allows the implementors to exploit any architecture specific features to improve performance. K42 supports the Linux API and ABI. The K42 group has been actively involved with the Research Hypervisor

Contact Information

Mailing List

Most queries should be directed to the mailing list. These include questions about building, running, or developing on or with K42. Difficulties encountered doing the above, should also be directed to the mail list.

Project Wiki

Please see the K42 Project Wiki for additional details on building K42, running K42, etc.

Report and Track bugs via Bugzilla

See the K42 Bugzilla page for reporting and tracking the status of bugs.

IM/IRC

An IRC channel exists for developers interested in live discussions and feedback: irc.freenode.net channel #k42

Specific Issues

Specific requests such as: linking this page to a project page, queries about employment or intern opportunities, or broken links discovered on this web page can be directed to:
email address in jpg to avoid spam

Mailing List
White Papers
Related Projects
University Ties
Publications
Presentations
The K42 Group
Open Source

Mailing List - Discussion Group

There is a public mailing for anyone interested in using K42. It has was started in 2003 and is designed for questions about K42. Questions about building K42, running K42, or developing on or with K42 are all welcome in this forum.
See: K42 Mail List and Discussion Group .

White Papers

Below is a series of white papers describing various aspects of K42's design. There is an overview paper in which we present at a high level K42's goals, motivations, design, and technologies of K42. There are also more targeted white papers. Many of the papers have an html and pdf version as well:

K42 Overview: postscript or pdf . (updated August 2002)
K42 Autonomic Computing in K42 via Hot-Swapping:
Paper with overall view and required system infrastructure: postscript or pdf . (updated November 2002)
Paper with more details about implementation and results: postscript or pdf . (updated November 2002)
K42's Performance Monitoring and Tracing: postscript or pdf . (updated August 2002)
Memory Management in K42: postscript or pdf . (updated August 2002)
Scheduling in K42: postscript or pdf ,. (updated August 2002)
Utilizing Linux Kernel Components in K42: postscript or pdf . (updated August 2002)
Building Blocks: A position paper (ps, pdf) presented at HOT OS 1997.

Related Projects

The Continuous Program Optimization (CPO) team has used K42 as a prototyping vehicle for developing the Performance and Environment Monitoring (PEM) infrastructure and for exploring scenerios for large page benefit analysis.

If you have a project you would like linked to from here please send email to:
email address in jpg to avoid spam

Also please place a link from your project's page to this one. Thank you.

University Ties

The K42 project is closely tied with the Tornado Operating System from the University of Toronto's Parallel Systems Group

Publications

: Orran Krieger, Marc Auslander, Bryan Rosenburg, Robert W. Wisniewski, Jimi Xenidis, Dilma Da Silva, Michal Ostrowski, Jonathan Appavoo, Maria Butrico, Mark Mergen, Amos Waterland, Volkmar Uhlig, "K42: Building a Complete Operating System", EuroSys 2006 , Leuven Belgium April 2006.
: Dilma Da Silva, Orran Krieger, Robert W. Wisniewski, Amos Waterland, David Tam and Andrew Baumann, "K42: An Infrastructure for Operating System Research", ACM SIGOPS Operating Systems Review , Vol. 40, no. 2, April 2006.
: Mark F. Mergen, Volkmar Uhlig, Orran Krieger, Jimi Xenidis "Virtualization for High-Performance Computing", ACM SIGOPS Operating Systems Review , Vol. 40, no. 2, April 2006.
: Andrew Baumann, Jonathan Appavoo, Dilma Da Silva, Jeremy Kerr, Orran Krieger, and Robert W. Wisniewski, Providing Dynamic Update in an Operating System", USENIX 2005 , pp. 279-291, Anaheim California April 2005.
Module Hot-Swapping for Dynamic Update and Reconfiguration in K42: Andrew Baumann, Jeremy Kerr, Jonathan Appavoo, Dilma Da Silva, Orran Krieger, and Robert W. Wisniewski, LCA 2005 linux.conf.au
K42: an Open-Source Linux-Compatible Scalable Operating System Kernel: pre-print version; Jonathan Appavoo, Marc Auslander, Maria Burtico, Dilma Da Silva, Orran Krieger, Mark Mergen, Michal Ostrowski, Bryan Rosenburg, Robert W. Wisniewski, Jimi Xenidis, "K42: an Open-Source Linux-Compatible Scalable Operating System Kernel", IBM Systems Journal pp. 427-440 Vol. 44, No. 2, 2005
Improving Operating System Availability With Dynamic Update: Andrew Baumann, Jonathan Appavoo, Dilma Da Silva, Orran Krieger, and Robert W. Wisniewski, "Improving Operating System Availability With Dynamic Update", OASIS (Workshop on Operating System and Architectural Support for the on demand IT InfraStructure) pps 21-27, October 9, 2004, Boston Masschusetts.
KFS: Exploring Flexibility in File System Design: Dilma Da Silva, Livio Soares, Orran Krieger, "KFS: Exploring Flexibility in File System Design", Brazilian Workshop on Operating Systems (WSO'2004), Salvador, Brazil, August 2004.
Meta-data Snapshotting: A simple mechanism for File System Consistency: Livio Soares, Orran Krieger, Dilma Da Silva, "Meta-data Snapshotting: A simple mechanism for File System Consistency", SNAPI'03 (International Workshop on Storage Network Architecture and Parallel I/O).
Efficient, Unified, and Scalable Performance Monitoring for Multiprocessor Operating Systems pdf or ps: Robert W. Wisniewski and Bryan Rosenburg, "Efficient, Unified, and Scalable Performance Monitoring for Multiprocessor Operating Systems", Supercomputing 2003.
Enabling Scalable Performance for General Purpose Workloads on Shared Memory Multiprocessors: Jonathan Appavoo, Marc Auslander, Dilma Da Silva, Orran Krieger, Michal Ostrowski, Bryan Rosenburg, Robert W. Wisniewski, Jimi Xenidis, Michael Stumm, Ben Gamsa, Reza Azimi, Raymond Fingas, Adrian Tam, David Tam, "Enabling Scalable Performance for General Purpose Workloads on Shared Memory Multiprocessors", IBM Research Report RC22863
Providing a Linux API on the scalable K42 Kernel: Jonathan Appavoo, Marc Auslander, David Edelsohn, Dilma Da Silva, Orran Krieger, Michal Ostrowski, Bryan Rosenburg, Robert W. Wisniewski, Jimi Xenidis, "Providing a Linux API on the scalable K42 Kernel", Proceedings of Freenix 2003, pg 323-336.
System Support for Online Reconfiguration: Craig A. N. Soules, Jonathan Appavoo, Kevin Hui, Robert W. Wisniewski, Dilma Da Silva, Gregory R. Ganger, Orran Krieger, Michael Stumm, Marc Auslander, Michal Ostrowski, Bryan Rosenburg, Jimi Xenidis, "System Support for Online Reconfiguration", Proceedings of Usenix 2003, pg 141-154.
Enabling Autonomic System Software with Hot-Swapping: pre-print version; Jonathan Appavoo, Kevin Hui, Craig A. N. Soules, Robert W. Wisniewski, Dilma Da Silva, Orran Krieger, Marc Auslander, David Edelsohn, Ben Gamsa, Gregory R. Ganger, Paul McKenney, Michal Ostrowski, Bryan Rosenburg, Michael Stumm, Jimi Xenidis, ``Enabling Autonomic System Software with Hot-Swapping.'' IBM Systems Journal, Vol. 42, No. 1, pg 60-76, 2003.
An Infrastructure for Multiprocessor Run-Time Adaptation: Jonathan Appavoo, Kevin Hui, Michael Stumm, Robert Wisniewski, Dilma da Silva, Orran Krieger, Craig Soules, ``An Infrastructure for Multiprocessor Run-Time Adaptation'', Proceedings of WOSS 2002 (ACM SIGSOFT Workshop on Self-Healing Systems), pg 3-8.
Position Summary: Supporting Hot-Swappable Components for System Software: full version; Kevin Hui, Jonathan Appavoo, Robert Wisniewski, Marc Auslander, David Edelsohn, Ben Gamsa, Orran Krieger, Bryan Rosenburg, Michael Stumm, ``Position Summary: Supporting Hot-Swappable Components for System Software.'' HOTOS 2001.
Tornado: Maximizing Locality and Concurrency in a Shared Memory Multiprocessor Operating System: Benjamin Gamsa, Orran Krieger, Jonathan Appavoo, Michael Stumm, ``Tornado: Maximizing Locality and Concurrency in a Shared Memory Multiprocessor Operating System.'' OSDI 1999, pp. 87-100
Customization-Lite: M. Auslander, H. Franke, O. Krieger, B. Gamsa, and M. Stumm, "Customization-Lite", In Proc. 6th Workshop on Hot Topics in Operating Systems (HotOS-VI), May, 1997, pp. 43-48.
HFS: A performance-oriented flexible file system based on building block composition: O. Krieger and M. Stumm, "HFS: A performance-oriented flexible file system based on building block composition", ACM Trans. on Computer Systems, August 15(3), 1997, pp. 286-321.

Related Publications

Multiple Page Size Modeling and Optimization: Calin Cacscaval, Evelyn Duesterwald, Peter F. Sweeney, and Robert W. Wisniewski "Multiple Page Size Modeling and Optimization", PACT 2005 (Parallel Architectures and Compilation Techniques) to appear.
Online Performance Analysis by Statistical Sampling of Microprocessor Performance Counters: Reza Azimi, Michael Stumm, and Robert W. Wisniewski, "Online Performance Analysis by Statistical Sampling of Microprocessor Performance Counters", ICS 2005 (International Conference on Supercomputing) to appear.
Performance and Environment Monitoring for Whole-System Characterization and Optimization: Robert W. Wisniewski, Peter F. Sweeney, Kartik Sudeep, Matthias Hauswirth, Evelyn Duesterwald, Calin Cascaval, and Reza Azimi, "Performance and Environment Monitoring for Whole-System Characterization and Optimization", PAC2 (Conference on Power/Performance interaction with Architecture, Circuits, and Compilers) 2004 to appear
relayfs: An Efficient Unified Approach for Transmitting Data from Kernel to User Space pdf or ps: Tom Zanussi, and Karim Yaghmour, Robert Wisniewski, Richard Moore, and Michel Dagenais, "relayfs: An Efficient Unified Approach for Transmitting Data from Kernel to User Space", OLS (Ottawa Linux Symposium) 2003 pg 519-531.
Efficient and Accurate Tracing of Events in Linux Clusters pdf or ps: Michel Dagenais, Richard Moore, Robert Wisniewski, Karim Yaghmour, and Tom Zanussi, "Efficient and Accurate Tracing of Events in Linux Clusters", HPCS (High Performance Computing Systems) 2003 .
Automatic Compiler-Inserted I/O Prefetching for Out-of-Core Applications: Todd C. Mowry, Angela K. Demke and Orran Krieger, "Automatic Compiler-Inserted I/O Prefetching for Out-of-Core Applications", Proceedings of the Second Symposium on Operating Systems Design and Implementation (OSDI '96), pages 3-17, October 1996.

Presentations

Slides for presentation on Scalability (6/18/2003): pdf postscript

The K42 Group

From IBM TJ Watson Research :: Jonathan Appavoo, Marc Auslander, Maria Butrico, Dilma DaSilva, Orran Krieger, Mark Mergen, Michal Ostrowski, Bryan Rosenburg, Volkmar Uhlig, Amos Waterland, Robert Wisniewski, and Jimi Xenidis.
Other IBM K42 contributors:
: Jeremy Kerr.
Other K42 code contributors:
: Reza Azimi, Andrew Baumann, Raymond Fingas.
Former K42'ers:
: Michael Britvan, Chris Colohan, Phillipe DeBacker, Khaled Elmeleegy, David Edelsohn, Cecilia Fernandes, Hubertus Franke, Ben Gamsa, Jan Gentil, Manjunath Gorentla Venkata, Garth Goodson, Darci Gurley, Kevin Hui, Edgar Leon, Craig MacDonald, Iulian Neamtiu, Michael Peter, Jim Peterson, Eduardo Pinheiro, Himanshu Raj, Bala Seshasayee, Rick Simpson, Marcio Santos, Livio Soares, Craig Soules, David Tam, Manu Thambi, Nathan Thomas, Gerard Tse, Timothy Vail, Chris Yeoh.

Open Source Code

K42 is an open source project. The code is available via our anonymous cvs respository, released under the following lgpl license. Please recognize our code is still in prototype form, and while it does run significant benchmarks there are still many corner cases not yet implemented. As of January 2004, a preliminary set of build and simulation tools are available. Some of our collaborators are running K42 on the Power Mac G5.

If you have any comments or questions please send email to the mailing list.

Page last updated April 2008 - if any links are broken please send email to:
email address in jpg to avoid spam

Attached file: Kittyhawk OSR 08.pdf

Last updated 9 Apr 2008

Research labs involved

Watson Research Center (Yorktown)