Usage DemonstrationLearn how to use TIES VM
TIES VM Usage Introduction
- 1 Introduction
- 2 Team
- 3 Process
- 3.1 Step 1: Log in to your Organization as Admin user
- 3.2 Step 2: Create New Users for Researcher and Honest Broker
- 3.3 Step 3: Representing Researcher – Search for DeIdentified Reports
- 3.4 Step 4: Representing Honest Broker – Search for Identified Reports
- 3.5 Step 5: Load and process your first batch of documents into Production
- 3.6 Step 6: Mark password for Reset and forward account information to Researcher and Honest Broker
- 4 Troubleshooting
- 4.1 What if my tomcat service can be reached from local computer but inaccessible to the outside world?
- 4.2 Are Pipe Processors installed and running?
- 4.3 Did The HL7ImportPipeController process the initial HL7 file?
- 4.4 Have Document.ApplicationStatus and IdentifiedDocument.ApplicationStatus been set to IDLING?
- 4.5 Are the Pipeline Log files clear of errors and indicative of successful runs?
- 4.6 Does the Production Index contain files? The Indexer Pipeline builds a Lucene Index at the designated area.
- 4.7 Fixes
- 4.8 Still having issues? See Troubleshooting for general troubleshooting help.
This page gives you brief introduction about how to use TIES on the VM.
Only the Administrator will exercise this plan. However, it is necessary that a Researcher, and Honest Broker be designated.
|Admin||Responsible for obtaining a license, installing and performing rudimentary QA.||The TIES Provider Admin must be able to create new Users and Studies, assign Users to Studies, and sanction requests from external studies to access local data and tissue. It is assumed the Admin will accomplish all the tasks in this Plan. Nevertheless, it is important that a Researcher and Honest Broker be designated.|
|Researcher||Individual designated as one of the first Primary Investigators at the Providing Organization.||This Researcher will have typically authored the original license request and will have been integral in the business plan to join TCRN.|
|Honest Broker||Individual designated as one of the first Honest Brokers at the Providing Organization.||May be a Tissue Bank manager, or LIS Data Management Supervisor.|
Node initiation proceeds in a sequence of steps. The next step does not begin until the previous step is completed and accepted.
Step 1: Log in to your Organization as Admin user
Access the TIES Client here. All new Administrators are given the same username “admin” and temporary password “Admin20!3”. Your VM is automatically populated with TCGA data. Begin by searching for these. Before searching, you need to establish a quality assurance study using your Admin privileges.
Step 2: Create New Users for Researcher and Honest Broker
Select the New User toolbar button. Fill in the information for your designated Researcher. Repeat the process for your designated Honest Broker. Take note of the username and auto-generated Password (here soar5cys). You will need to enter this information when acting as the Researcher and Honest Broker delegate.
Step 3: Representing Researcher – Search for DeIdentified Reports
Login as your Researcher using the temporary password. Change the password, remembering it for later. A good idea might be to consistently just add one letter to the original auto-generated password. Click “Pathology” in the reports type search criteria and return. The standard test corpus is 10,000 documents all of which are Pathology. The query payload should match this number. Check a single document result to verify that most of the system is working.
Step 4: Representing Honest Broker – Search for Identified Reports
Login as your HonestBroker using the temporary password. Change the password, remembering it for later. You will need to be inside your Organizational Intranet to see reports. But after issuing a query you should see reports that contain identified patient health information and contain no run message of the DeIdentifier. Since this is test data the names of the patients will be fictitious (e.g., John and Jane Doe).
Step 5: Load and process your first batch of documents into Production
If all went well with your installation, you should have a series of background processes running that represent individual phases of the TIES ETL process (see Loading Your Data). Specifically, you will have a process for:
- Concept Coder
These processes are designed to run automatically on system startup but may not be running right after starting the VM. You can start them with the command sudo ties-pipeline start at /home/ties/TIES/bin/. They poll for a change of state, do their work, and then sleep for a predefined period. Change of state usually indicates that there are documents ready to be worked upon. You have an option to use different De-identifiers with TIES. The DeidPipe configuration file is located here: /home/ties/TIES/pipelines/DeidPipeController/classes/config/caTIES.properties Load your first HL7 file We assume your AP LIMS can output Report data in HL7 format. Separate a small but representative initial set of HL7 records into a separate file. Then copy this file into the HL7Loader input directory. The default location for this directory is here in the installation footprint. It is set in /home/ties/TIES/pipelines/HL7ImportPipeController/classes/config/caTIES.properties. Look for the attribute value pairs associated with the HL7ImportPipeController. caties.hl7importer.directory.home = /home/ties/TIES/HL7 caties.hl7importer.document.type = PATHOLOGY caties.hl7importer.sleepsize = 300000 caties.hl7importer.ethnicity.cfg = config/EthnicityConfig.txt caties.hl7importer.gender.cfg = config/GenderConfig.txt caties.hl7importer.race.cfg = config/RaceConfig.txt #Uncomment following line to tag all incoming reports with the specified tags. #Tags cannot contain spaces or colons in their name. #caties.hl7importer.tag = testtag Once the HL7Loader pipe is complete it will write the HL7 file into the processed directory.
Step 6: Mark password for Reset and forward account information to Researcher and Honest Broker
This is standard TIES procedure after you add a new user. The only difference here is that you have used these accounts to benchmark the installation process. This was necessary because TIES requires each study to have a Primary Investigator (Researcher) and an Honest Broker. We also discourage the use of fictitious user information polluting the production database. Thus the temporary delegation is one means to an end. Another solution might be to assign yourself as Administrator the additional roles of Researcher and Honest Broker and make two Studies to exercise the public and private data search and order set construction. When the actual Researcher and Honest Broker log in for the first time, they will be asked to change their passwords.
If no reports are found, it is likely that the data transformation was clogged in one of the pipes. To troubleshoot this situation there are a number of techniques and tools available.
What if my tomcat service can be reached from local computer but inaccessible to the outside world?
Try adding tomcat.exe to the Windows firewall’s exception list. If it still does not work, you might need to contact the network people at your institution.
Are Pipe Processors installed and running?
First check to see if the pipeline processors are installed and running. In Windows this can be accomplished via Control Panel -> All Control Panels -> Administrative Services -> Services. All TIES sevice names are prefixed with “TIES.” They should be started by the user who installed TIES initially (the Administrator).
Did The HL7ImportPipeController process the initial HL7 file?
It should have been moved to the processed sub directory.
Have Document.ApplicationStatus and IdentifiedDocument.ApplicationStatus been set to IDLING?
The TIES acquisition phase is governed by a two MySQL relational databases. Check the status of ties_private.IDENT_DOCUMENT and ties_public.DOCUMENT to see that all Documents are present and in APPLICATION_STATUS = IDLING. You may use either the command line interface of MySQL, download the MySQL Workbench Visual Editor or use any available RDBMS tool like Squirrel. select application_status, count(*) from document group by application_status; State Transition for the ETL process should be as follows:
If the application_status of some documents never make it to IDLING there is typically an error in the process. Use log files and TIES support staff to help solve the issue.
Are the Pipeline Log files clear of errors and indicative of successful runs?
Each Pipeline has two log files in <TIES INSTALL PATH>/pipelines/<PipeName>/logs/service.log and <TIES INSTALL PATH>/pipelines/<PipeName>/logs/wrapper.log Check these for diagnostics of a good run as well as any Java stack traces indicative of a failed run.
Does the Production Index contain files? The Indexer Pipeline builds a Lucene Index at the designated area.
This index should be a non empty directory of binary files.
If it is determined that the production data transformation and load has failed you should:
- Examine the status of each pipeline by sudo ties-pipeline status
- If you are unable to see the windows services running then, run the command to start each pipeline sudo ties-pipeline start
- Examine the parameters in the caties.properties file in each pipeline footprint
- Forward error messages and captured stack traces to TIES Support Staff