Subject Infrastructure Repository

Get Access!
SIR Software Objects
SIR Users and Publications
Frequently Asked Questions
Repository License

These pages provide information on and access to the Software-artifact Infrastructure Repository (SIR) for Experimentation. This repository provides Java, C, C++, and C# programs for use in experimentation with testing and analysis techniques, and materials facilitating that use. Descriptions of the motivations, origanization, and objectives of the repository can be found in the journal paper.

The repository is still being maintained, and may be expanded further in the future. If you have suggestions regarding how we can improve the repository to make your experience better, we want to hear from you at Your feedback is appreciated.

Use the links on the navigation bar at left to obtain materials. The "Handbooks" describe how objects are created and provide information on their use. The "Download Objects" page provides links to tar files containing actual objects. "Biographies" of the objects are available from the download page by clicking on program names. "Tools" are various scripts and programs that help process object materials.

Choosing to download these object programs implies acceptance of our SIR User License.


  • 9/6/2018 The Sofya tools site has moved to GitHub! While still limited to Java 7 and earlier, we no longer require login to users to request access.
  • 2/14/2014 The xml-security object has been updated to fix a problem with the junit test scripts lacking setUp() blocks. This also contains fixes that would have impacted building the v3 version.
  • 6/20/2013 The SIR will be unavailable on Friday, June 21, 2013 from 6:00 AM CDT to 12:00 PM CDT for server maintenance. We apologize for this inconvenience.
  • 5/23/2013 Several new artifacts for studying testing of software product lines (SPL) have been added to the repository.
  • 3/22/2013 The SIR team announces support for a new artifact language family, C++, and our first testing subject in C++, concordance, offering a rich set of mutated and inserted faults along with an extensive test suite.
  • 7/25/2012 A new version of the mts tool package has been uploaded which includes a tool for creating mono test scripts (shell scripts) from universe (STIMPL) files.
  • 5/8/2012 An updated version of the jtcas object has been installed. The fault in version 21 was incorrectly placed in the wrong method/function in the 1.0 version.
  • 4/1/2012 New concurrency testing subjects are available for download.
  • 12/19/2011 The SIR repository will have maintenance outage starting on December 23, 2011 and extending through December 27, 2011 as our host server is upgraded. The SIR will be unavailable during this time. We apologize for any inconvenience this may cause.
  • 11/22/2011 The nanoxml object has been redelivered to the repository to now includes the UML state diagrams for the package classes.
  • 07/14/2011 The problem of missing classes has been remedied in all the Java invariant objects (Stack, Red-Black Tree, Disjoint Set, Binary Heap, Binary Search Tree, Vector, Sorting, Doubly-Linked List, and Array Partition.) All references to the missing classes have been changed to standard Java classes or have been removed. The test classes provided have also been removed, since those classes required using the unavailable classes. If you wish to download the original versions of the packages (which still include the original test classes) you may do so via the Archived Objects link in the left panel.
  • 06/01/2011 A port of the Siemens tcas to the C# language is now available for download. This tcas-csharp object is similar to the jtcas port of tcas to Java which is also now available.
  • 05/09/2011 The problem with downloading using Chrome and Opera browsers has been fixed.
  • 05/07/2011 A problem with downloading using the Chrome and Opera browsers has been identifed. The download object links work reliably with Firefox 3.6, IE 8 and 9, and Safari 5.0.5 on Mac OS X. We will report further on this issue and we appologize for the inconvenience.
  • 3/15/2011 The nanoxml object has been redelivered to correct the extended fault matrices which were incorrect. Also one fault has been removed from those provided in nanoxml v5 which was overly expressive according to SIR fault guidelines. The fault can still be activated but the fault matrix does not include this fault.
  • 2/10/2011 It has been brought to our attention that all invariant objects specify several classes that are not provided with the packages and cannot be built by SIR users. This applies to the objects: Stack, RedBlackTree, BinarySearchTree, ArrayPartition, DoublyLinkedList, Sorting, Vector, BinaryHeap, and DisjointSet. We have contacted the original institution to obtain these classes. An announcement will be given on this page when this oversight has been rectified.
  • 9/23/2010 An updated version of nanoxml has been produced which contains 21 additional test frames to increase test coverage.
  • 11/9/2009 The SIR team announces the availablity of Java concurrency testing subjects to the repository. These subjects have been created and used by several research groups for empirical studies into path sensitive Java constructs and model checking analyses. These test subjects have been packaged for use with the JavaPathFinder model checking environment.
  • 10/6/2009 An updated version of xml-security has been delivered into the repository with object preparation bugs fixed. A summary of the bug fixes are: wrong/bad version of xercesImpl.jar in the seeded common libs directory, a non-functional/unnecessary #include in XMLSignatureInput.cpp, and an incorrect seeding (#ifdef) of XU_HD_2 in XMLUtils.cpp. Our thanks to Jake Cobb at Georgia Tech for finding these.
  • 9/22/2009 The SQUALL lab at Carleton University has contributed the Cruise Control, Elevator and Ordered Set (OrdSet) testing research objects. These objects feature state machine code block elements making them well suited to test coverage and minimalization studies on finite state machines.
  • 8/10/2009 The Derby DB subject has been released from SIR development for use as a testing research object. Derby has over 500 KLOC and inserted faults making it well suited to studies of database systems testing and medium to large size project testing.
  • 2/2/2009: Researchers from the SAnToS Group at Kansas State University have contributed a set of Java heap-based data structure examples for the purpose of evaluating symbolic execution and unit-test generation tools for Java.
  • 7/24/2008: We are proud to announce beta version availability of two new artifacts to the repository. The bash (Bourne Again Shell) and vim (vi Improved) objects are now available for experimentation.
  • 10/31/2006: Bug fixes have been released for make. The fixes resolve various problems with the STImpL (universe) files that caused test scripts to behave incorrectly. Please consult documentation for additional details.
  • 10/23/2006: Bug fixes have been released for JMeter and xml-security. The fixes resolve a problem with library jar file dependencies causing javac to crash. For the curious, the error appears actually to be in the Sun library classes for handling jar files (a bug report has been filed), however repackaging the jar files seems to suffice as a workaround; e.g.:
      jar -xf <jar_file>
      rm <jar_file>
      jar -cf <jar_file> <jar_contents>
  • 10/02/2006: We are happy to announce a beta release of the reworked sed object. The new version of sed now provides a comprehensive test suite designed according the best-practice TSL principles. It also includes new versions, as well as developer provided tests, and basic information about real faults in old versions. As a beta release, we request your patience in the event that you encounter issues with the object, and we want to hear about any such issues.
  • 10/02/2006: A bug fix to grep has been released to fix compile errors that have been encountered on more recent versions of the gcc compiler.
  • 09/19/2006: A new line counter for C objects has been added to the SIR tools downloads. This line counter accumulates counts of the non-comment physical lines of source code in programs written in C (among numerous other languages). We note that line counts can vary greatly depending on the tool used and strategy employed. For this reason, and for consistency, we have run this line counter on our existing C objects, which has revised some of the numbers reported for these objects.
  • 09/14/2006: If you have been using C objects with the latest version of GCC, you may have noticed some compilation errors. We are aware of this problem and are working on fixes.
  • 05/04/2006: New version of mts/javamts is available. Version 1.1 adds an XML file generator, contributed by Ben Liblit of University of Wisconsin, and other minor fixes.
  • 02/17/2006: A new Java implementation of mts/javamts is now available in the tools download, replacing the old mts implementation. The new implementation features an improved interface and a number of enhancements, while still supporting backward compatibility. Please refer to the README file in the download package for details.


The SIR project was conducted in two phases. Phase 2 of the SIR project was supported by the National Science Foundation under Grants CNS-0958231, CNS-0958199, CNS-0958235, and CNS-0958346 .

Phase 2 Lead Personnel were:

  • Sarfraz Khurshid, University of Texas - Austin
  • Darko Marinov, University of Illinois, Urbana-Champaign
  • Gregg Rothermel, University of Nebraska - Lincoln
  • Tao Xie, North Carolina State University
  • Wayne Motycka, University of Nebraska - Lincoln

Phase 1 of the SIR construction effort was supported by the National Science Foundation under Grants 0454203 and 0454348 .

Phase 1 Lead Personnel were:

  • Matthew B. Dwyer, University of Nebraska - Lincoln
  • Sebastian Elbaum, University of Nebraska - Lincoln
  • John Hatcliff, Kansas State University
  • Gregg Rothermel, University of Nebraska - Lincoln
  • Hyunsook Do, University of Nebraska - Lincoln
  • Alex Kinneer, University of Nebraska - Lincoln

Many other students, faculty, and researchers from various institutions and companies have also been involved in or aided the construction of this repository or the objects provided through it, and we wish to thank them:

Adam Ashenfelter Yuriy Brun Sean Callan
Soumya Chattopadhyay Dan Chirica Chengyun Chu
Desiree Dunn Mike Ernst David Friberg
David Gable Amit Goel Madeline Hardojo
Mary Jean Harrold Angela Holland Binh Huynh
Dalai Jin Praveen Kallakuri Satya Kanduri
Srikanth Karre Alex Kinneer Jim Law
Khoa Le Alexey Malishevsky Kyle Murphy
Alex Orso Sameera Reddy David Rosenblum
Joe Ruthruff Roland Untch Ted Whaler
Vasanth Williams Rui Wu Weiyun Wu
Zhiru Zhu Tom Ostrand Phyllis Frankl
Filip Vokolos Monica Hutchins Herb Fisher
Tarak Goradia Zhimin Wang Isis Cabral
Suzette Person Samar Mouchawrab Zhihong Xu

[†] Any opinions, findings, and conclusions or recommendations expressed in this material are those of the author(s) and do not necessarily reflect the views of the National Science Foundation.


Try the following link to upgrade the page display. (Explanation)