NIEM Newsletter
NIEM Tools: An In-Depth Look
By Marc Clifton
Although it might be possible to build something with just bare hands and ingenuity, it always helps to have a few tools in your toolbox to make the job easier and more efficient. The same goes for working with the National Information Exchange Model (NIEM). A number of tools are available to help with NIEM, many of which are included on the www.niem.gov Web site. In this article, we will review the following tools and discuss what they do, why they are needed, and how you can use them:
- Search and navigate NIEM (part of the Subset Schema Generation Tool [SSGT]).
- Build Schema Subset using SSGT.
- Map Information Exchange.
- Work With Information Exchange Package Documents (IEPDs).
- Generate Code List Schemas.
- Obtain Migration Assistance.
Note: Bear with me in the listing of the tools, since they do not have catchy names or succinct titles. Many are named simply for what they do; this can assist in immediate recognition of their value but also makes for awkward article subheadings.
Search and Navigate NIEM
Clicking on this link takes you to the NIEM SSGT. I will discuss the schema subset generation tool later—for now, we will look just at the search and navigation features. The default Web page provides the basic search capability, allowing you to search by:
- Property
- Type
- Namespace
- Facet
- External
- Association
The most common searches are for "Property" and "Type." An advanced search option brings up a user interface in which you can specify advanced options, such as searching for the exact phrase, what you want to match on, and which domains you want to search.
At some point during the IEPD creation process, you will most likely create a Unified Modeling Language (UML) diagram of your logical model. Once the logical model is created, you will want to search NIEM for the semantic equivalent types (classes) and elements (properties) of your UML model so they can be mapped to NIEM schema types and elements. Further, you will discover NIEM types that are not semantically equivalent but are suitable to extend or substitute for a custom type (which goes into your extension schema) required in your exchange.
For example, to find whether NIEM contains a "Citation" type, we can change the "Search for a" to "Type," enter the word "Citation" in the search box, and click on the Search button. This yields a single result, "j:CitationType." At this point, we can drill into the elements of the CitationType by clicking on the plus sign to the left of the "Add" button. You can drill further into the subelements, and so forth, by navigating into the hierarchy. Similarly, to see what type j:CitationType inherits, you can click on the text "show inheritance," which reveals that j:CitationType inherits from nc:ActivityType. Finally, next to each element is the text "details"; when you click on this text, it shows the definition for that element. This is very valuable in checking whether the NIEM type has semantic equivalence to the desired type in the exchange.
Building a Schema Subset Using the Schema Subset Generation Tool (SSGT)
CitationIssuedLocation is an element of j:CitationType, j:CitationType is automatically added to the subset schema, whereas j:CitationIssuedLocation was not automatically added when adding nc:LocationAddress. This is because the type containing the element nc:LocationAddress is nc:AddressType, whereas the type containing element j:CitationIssuedLocation is j:CitationType. When a subset schema is generated, the containing type is always included.
When adding elements, you can specify the cardinality right away by clicking on the down arrow next to the "Add" button. This provides a list of cardinality options and will generate the appropriate "minOccurs" and "maxOccurs" attributes for that element. The desired cardinality is usually determined by your model and should be documented in the UML model you are using as a reference for mapping to NIEM. You can, however, defer selecting the cardinality until later. Once your subset schema is created, you can click on the "Edit Cardinality" link to define the cardinality for all elements. You can also delete types and elements by clicking on the checkbox to the left of the type/element and then clicking on the "Delete" button.
Creating a large subset schema probably will not be accomplished in a single session, and it may be the work of several people as well. The SSGT lets you save the schema subset as a "wantlist," allowing you (or others) to return to your work later on. The "Save current wantlist to a file" is available on the "Generate Documents" page. You can later load your wantlist from the "Options" link on the main page of the SSGT, where you will find the section "Load Wantlist."
Map Information Exchange
The mapping tool helps you create the exchange and extension schemas and map your UML model with NIEM. It will also generate the subset schema for your exchange. To use this tool, you will need to generate a UML 1.4 or UML 2.1 metamodel in XML Metadata Interchange (XMI) representation using a tool such as ArgoUML or Eclipse. Once you have uploaded the XMI model, you can use this tool to map the classes and properties in your UML to NIEM. While this tool provides NIEM search capabilities, it is less sophisticated than the SSGT navigation and search, so you may want to have a separate browser tab or window open to the SSGT. When you locate a type or element using the mapping tool, you have the option to use this tool, create an exchange on the main Web page, and then upload one or more XMI models. You can then begin mapping the exchange by clicking on the XMI model link.
Classes and their derivatives are presented as root-level artifacts in a navigation tree. For example, if you load the XMI for the Citation UML, the navigation tree displays "DocumentType" and "CitationBatch." Both can be expanded to show their properties. Classes that are part of the UML model by association are not directly represented; instead, the properties of these associative classes can be mapped by navigating the association described in the UML model. Therefore, it is important that you label your associations (some UML modelers allow for unnamed associations) because the association names will appear in the navigation tree of the mapping tool. If the association name is missing, the mapping tool displays the associated class name. So, when working with the mapping tool, remember that it is association-centric. Whereas the SSGT will explicitly show you the containing types (for example, adding j:CitationType automatically brings in nc:ActivityType), the mapping tool will not explicitly show this to you. However, it behaves the same as the SSGT, so when you map Citation to j:Citation, the niem-core subset schema will also include nc:ActivityType. This brings up another difference between the mapping tool and the SSGT: with the mapping tool, you are mapping properties to elements.
Once you have mapped your UML classes and properties, you can generate the subset schema, extension schema, and exchange schemas by clicking on the "here" in the text "To generate and download a mapping report, click here." This will take you to a page where, near the bottom, you can click on the text "generate an exchange schema." From this page, you may also create a mapping report, mapping set, wantlist, subset schema, and constraint schema.
Work With IEPDs
With this tool, you can upload or enter the schemas and documentation required for an IEPD. Along with the metadata, the tool assembles everything in a package that conforms to the NIEM IEPD specification. With this tool, you can also validate that the minimum artifacts and metadata are present for a NIEM-conformant IEPD. You can work with several IEPDs at once in various stages of completion. Once the IEPD has been created, you can make the IEPD, or parts of the IEPD, publicly accessible. You may also download IEPDs that other people have shared.
Begin with signing in, since IEPDs are specific to your account. Once you have signed in, you can create a new IEPD by uploading an IEPD zip file or by creating one using the tool. When creating a new IEPD, you will be asked to specify the root directory name and will be able to upload various artifacts for that IEPD.
The artifact list is quite extensive and includes artifact types such as business requirements and rules, constraint, subset, extension and exchange schemas, various documentation types, and so forth.
After specifying the artifacts (this does not have to be a complete list), you will be asked to enter some basic information about the IEPD.
The last step offers you the option to validate the IEPD and/or upload the IEPD artifacts. The validation does not validate the artifacts—instead, it validates the information in the metadata, such as version, Uniform Resource Identifiers (URI), domain, authoritative source point of contact, etc. You can always add or edit your artifacts by clicking on the "Edit" link and then selecting one of the options: Edit Metadata and Artifacts, Edit Visibility/Sharing, or delete the IEPD.
You may also download the IEPD. This includes all of your artifacts; in addition, the download includes the file "metadata.xml," which describes your metadata entered on the tool's Web pages as well as a catalog.html file. The catalog file can be viewed in your Web browser and provides links to various artifacts. Most browsers support viewing the XML/XSD files directly in the browser. (Note that Google's Chrome shows only the comments—it appears to strip off the XML tags in XSD and XML files.)
If you wish, you can make your IEPD publicly available by registering it with the IEPD Clearinghouse. You can also search public IEPDs by clicking on the "Search" link in the "work with IEPDs" section of the tool's Web page.
Generate Code List Schemas
This tool generates a code list (eye color, hair style, etc.) from an Excel spreadsheet. Code lists are commonly used as substitution groups along with generic text entry and represent a pick list of valid options for a particular element. To create your own code list, download the template zip file. When you extract it and open the XLS document, note that there are two tabs with sample code lists. You can replace/remove/add new tabs as appropriate. (The nice thing about the tool being tab-aware is that you can manage all of your code lists in a single spreadsheet.)
When you are ready to generate your code list, fill in the information presented on the tool's page and click the "Build Schema" button (on my browser, the word "Schema" is truncated). After the code list schema is generated, your browser will open a window to save the file locally. This will include your code list schema, the XLS file you uploaded, and the necessary NIEM artifacts to support the code list, such as the structures schema.
Obtain Migration Assistance
The migration assistance tool can convert a Global Justice XML Data Model (GJXDM) 3.0.3 wantlist to a NIEM 2.0 wantlist or a NIEM 1.0 wantlist to a NIEM 2.0 wantlist. The resulting output will be a NIEM 2.0 wantlist, a NIEM 2.0 subset schema, and a migration report. The migration report contains actions taken and choices made in migrating the wantlist, issues that cannot be resolved automatically, and statistics indicating degree of migration resolution. This tool cannot migrate extension or constraint schemas.
To use this tool, specify the GJXDM 3.0.3 or NIEM 1.0 wantlist file and the wantlist version, and then click on the "Migrate" button. The generated documents can then be saved locally for your review.
Conclusion
When creating an IEPD and mapping your logical model to NIEM, you will find the search and navigation tools indispensible. Most IEPDs usually have code lists that are specific to the domain—creating these code lists with the code list generator tool is a simple process of editing an Excel spreadsheet to generate the code list artifacts. The mapping tool itself is very useful in creating the subset, extension, and exchange artifacts required by the IEPD. These artifacts, supplemented with other artifacts (such as documentation, examples, and metadata) using the "Work with IEPDs" tool, provide a comprehensive tool for managing and publishing the IEPD.
Marc Clifton (www.marcclifton.com) is a .NET developer, consultant, and NIEM Practical Implementer's Course instructor. Much of the information from this article comes from the NIEM Practical Implementer's curriculum. For more information about the NIEM Practical Implementer's Course, visit www.niem.gov/training.php.
NIEM Case Study Office of Foreign Assets Control Specially Designated Nationals List Information Sharing
Synopsis
As part of the ongoing efforts being made by the U.S. Treasury's Office of Foreign Assets Control (OFAC) to integrate the National Information Exchange Model (NIEM) into its enterprise, the Specially Designated National (SDN) list data is being converted to a format that uses NIEM data elements. OFAC's completion of its NIEM-SDN pilot project has revealed the utility of a scalable, universal exchange format, and the hope is that future exchange relationships with other agencies will consume fewer resources and be completed in shorter periods of time. OFAC is already seeing a benefit from its use of NIEM and its introduction of NIEM-centered data rules. All-new SDN-related data elements introduced into OFAC systems are required to be NIEM-compliant. This has eliminated the discussion and debate that often surrounds every new data element and is already saving OFAC time and money.
Agency Overview
The U.S. Department of Treasury's Office of Foreign Assets Control administers and enforces economic and trade sanctions based on U.S. foreign policy and national security goals against targeted foreign countries and regimes, terrorists, international narcotics traffickers, those engaged in activities related to the proliferation of weapons of mass destruction, and other threats to the national security, foreign policy, or the economy of the United States. OFAC acts under Presidential national emergency powers, as well as authority granted by specific legislation, to impose controls on transactions and freeze assets under U.S. jurisdiction. Many of the sanctions are based on United Nations and other international mandates, are multilateral in scope, and involve close cooperation with allied governments.
Challenge
The Office of Foreign Assets Control (OFAC) is responsible for administering and enforcing the U.S. economic sanctions policy. As part of this mission, OFAC maintains a list of Specially Designated Nationals (SDN). All U.S. persons are prohibited from dealing with the individuals and entities on this list, which can include other government agencies as well as individuals. This list needs to be made available to a number of communities—specifically the trade, financial, and government sectors—in a useful data format, but, to date, OFAC has not been able to find a single format that can be used by these different communities without custom modifications of that format.
Solution
OFAC had been looking for a universal data standard for some time but was unable to locate a standard mature enough to meet the needs of many federal agencies. NIEM was the first standard that met OFAC's requirements in this regard.
OFAC began using the NIEM mapping tools on the NIEM Web site in 2007. Over a period of approximately one year, the office converted its SDN list standard into a UML model that could be used with the NIEM mapping tools. Once it was determined that the majority of NIEM data elements could be substituted for current OFAC SDN data elements, OFAC began constructing its IEPD and its exchange model. As a final proof of concept, OFAC manually converted all of its SDN data elements to the appropriate NIEM data elements using its newly constructed exchange map. Upon completing this conversion, OFAC began work on a series of scripts that would process existing SDN data into a NIEM-compatible format.
Results
The ultimate results of the OFAC-to-NIEM conversion effort are still playing out. However, the most immediate benefit has been the introduction of the NIEM data element library internally. OFAC no longer has to guess the nomenclature and data types of new data elements that are added to its systems; OFAC simply uses the appropriate NIEM elements.
OFAC's completion of its NIEM-SDN pilot project has revealed the utility of a scalable, universal exchange format. It is OFAC's hope that future exchange relationships with other agencies will consume fewer resources and will be completed in shorter periods of time. OFAC is already seeing a benefit from its use of NIEM and its introduction of NIEM-centered data rules. All-new SDN-related data elements introduced into OFAC systems are required to be NIEM-compliant. This has eliminated the discussion and debate that often surrounds every new data element and is already saving OFAC time and money.
NIEM PMO Call for Papers and Participation
The NIEM National Training Event will be held in Baltimore, Maryland, from September 30 to October 2, 2009. There will be six concurrent tracks focusing on three major topic areas—program management, architecture, and implementation. The NIEM Program Management Office (PMO) is seeking papers that will support the nationwide implementation and use of NIEM in a variety of domains, including justice, public safety, homeland security, health and human services, maritime, courts, and corrections. The Call for Papers and Participation is open to practitioners, the private sector, and educators, and the deadline for submission has been extended to April 15, 2009.
Registration Is Open for NIEM National Training Event
Registration for the 2009 NIEM National Training Event is now available online, via fax, and by mail. Please visit the event Web site for access to all available information on this event, including online registration, track descriptions, and hotel information. If you have any questions on the 2009 NIEM National Training Event, please send an e-mail to information@niem.gov.
Featured FAQ: What Is an IEPD and How Do I Use One?
Information Exchange Package Documentation (IEPD) is a set of data artifacts used to support the sharing of data for a particular business purpose. It is a set of documentation that accurately and completely defines the contents of a specific information exchange. For additional information on what makes up an IEPD and how to use one in an information exchange, please refer to the NIEM Concept of Operations.
NIEM Training News
NIEM Practical Implementer's Course-Department of Motor Vehicles, Carson City, Nevada
A NIEM Practical Implementer's Course was held on March 17-19, 2009, in Carson City, Nevada. This session was hosted by the Nevada Department of Motor Vehicles (DMV) and was attended by 17 students. Course participants included representatives from the Nevada DMV, the Nevada Department of Public Safety, and the Nevada Division of Child and Family Services, as well as industry representatives from Advantage Factory and Systems Made Simple. The combination of state agencies and industry representatives provided the setting for an engaging and informative training session. The instructors for this training engagement were Aaron Gorrell and Di Graski. The class was well-received based on the student evaluations.
NIEM Practical Implementer's Courses are funded through grants from the Bureau of Justice Assistance, U.S. Department of Justice, in cooperation with the U.S. Department of Homeland Security. For more information about the NIEM Technology Training Program, contact training@ijis.org.
Upcoming Events and Training
April 14-16, 2009: NIEM Practical Implementer's Course, Ashburn, Virginia
April 15, 2009: Deadline for NIEM National Training Event Call for Papers and Participation
September 30-October 2, 2009: NIEM National Training Event, Baltimore, Maryland Register Now
