Skip to main content

California Digital Library Announces New Release of the eXtensible Text Framework (XTF)

May 15, 2008 Author: Lisa SchiffCategories:

By Lisa Schiff, Technical Lead for CDL Publishing Services

The California Digital Library (CDL) is pleased to announce a new release of its search and display technology, the eXtensible Text Framework (XTF) Version 2.1.   XTF is an open source, highly flexible software application that supports the search, browse and display of heterogeneous digital content.  XTF offers efficient and practical methods for creating customized end-user interfaces for distinct digital content collections.

Highlights from the 2.1 release include:

  • Extensive interface improvements, including new search forms, built-in faceted browsing, and new look and feel.
  • Increased support for document and information exchange formats.
    • XHTML and OAI-PMH output
    • NLM article format indexing and output
    • Microsoft Word indexing
  • Streamlined XSLT stylesheets for simpler deployment and adaptation.
  • Updated documentation that has been moved to the XTF project wiki, allowing XTF implementers to share solutions with entire user community.
  • “Freeform” Boolean query language, offered as an experimental feature.
  • Backward compatibility with existing XTF implementations.

A complete list of changes is available on the XTF Project page on SourceForge, where the distribution (including documentation) can also be downloaded.

Since the first deployment of XTF in 2005, the development strategy has been to build and maintain an indexing and display technology that is not only customizable, but also draws upon tested components already in use by the digital library and search communities – in particular the Lucene text search engine, Java, XML, and XSLT.  By coordinating these pieces in a single platform that can be used to create multiple unique applications, CDL has succeeded in dramatically reducing the investment in infrastructure, staff training and development for new digital content projects.

XTF offers a suite of customizable features that support diverse intellectual access to content.  Interfaces can be designed to support the distinct tools and presentations that are useful and meaningful to specific audiences.  In addition, XTF offers the following core features:

  • Easy to deploy: Drops directly in to a Java application server such as Tomcat or Resin; has been tested on Solaris, Mac, Linux, and Windows operating systems.
  • Easy to configure: Can create indexes on any XML element or attribute; entire presentation layer is customizable via XSLT.
  • Robust: Optimized to perform well on large documents (e.g., a single text that exceeds 10MB of encoded text); scales to perform well on collections of millions of documents; provides full Unicode support.
  • Extensible:
    • Works well with a variety of authentication systems (e.g., IP address lists, LDAP, Shibboleth).
    • Provides an interface for external data lookups to support thesaurus-based term expansion, recommender systems, etc.
    • Can power other digital library services (e.g., XTF contains an OAI-PMH data provider that allows others to harvest metadata, and an SRU interface that exposes searches to federated search engines).
    • Can be deployed as separate, modular pieces of a third-party system (e.g., the module that displays snippets of matching text).
  • Powerful for the end user:
    • Spell checking of queries/li>
    • Faceted displays for browsing
    • Dynamically updated browse lists
    • Session-based bookbags

These basic features can be tuned and modified.  For instance, the same bookbag feature that allows users to store links to entire books, can also store links to citable elements of an object, such as a note or other reference.

A sampling of XTF-based applications both within and outside of the CDL include:

  • Mark Twain Project Online (http://www.marktwainproject.org), developed by the Mark Twain Papers Project, the CDL and the University of California Press.
  • Calisphere (http://www.calisphere.org/), a curated collection of primary sources keyed to the curriculum standards of California’s K-12 community, developed by the CDL.
  • The Encyclopedia of Chicago (http://www.encyclopedia.chicagohistory.org/), developed by the Chicago History Museum, The Newberry Library, and Northwestern University.
  • The Chymistry of Isaac Newton (http://webapp1.dlib.indiana.edu/newton/) and The Swinburne Project (http://webapp1.dlib.indiana.edu/swinburne/www/swinburne/), Indiana University.
  • Finding Aides at the New York Public Library (http://labs.nypl.org/2007/10/30/extensible-text-framework-xtf/).
  • EECS Technical Reports (http://sunsite2.berkeley.edu:8088/xtf/servlet/org.cdlib.xtf.crossQuery.CrossQuery?rmode=btr), UC Berkeley.