Auto Import and Auto Export

Auto Import connects to third party services such as FTP or AWS S3 to download documents and automatically add them to Cases. Auto Export connects to third party services such as FTP or AWS S3 to automatically export documents. ClearLife Limited will setup Auto Import and Auto Export for you and then you can view scheduled imports and exports by selecting Auto Import & Auto Export from the Admin menu.

Setup Auto Import and Auto Export

Contact ClearLife to setup Auto Import and Auto Export for you. See Contact Us.

We need:

• Server Address, like ftp://ftp.onehub.com or ftp://myftp.sharefileftp.com for FTP
• FTP Security and Protocol: there are several ways to securely transfer files and they can be confusing. The choices are:
  • FTP, explicit over TLS/SSL
  • FTP, implicit over TLS/SSL
  • SFTP, SSH File Transfer Protocol
  • Plain FTP is not recommended as authentication and data transfers are not encrypted.
  • FTP Over SSH is not supported.
• Username and password. Do not use ClearLife's individual team name, it is better to use a login name and password you control, not ClearLife. In the future, you will set the username and password yourself.
• Notification Emails: after each run, we send an email with a Summary Log showing which files were imported to which Case.
• Frequency: how often you want to run the Import. Typically, daily in the middle of the night.

Auto Import Usage

To see the status of Auto Import, select Upload Data from the Cases menu.

You can have multiple Auto Import tasks, each line in the summary table shows the last status under Last Import Result, start and finish times, as well as a download link for the Last Summary Import Log:

Click the magnifier to get full details of the current and previous runs, as well as download full logs:

The Last Summary Import Log is emailed to the Notification Emails after each run.

It is good practice to check daily for errors shown in the Summary Log/Email and fix them. Do not allow errors to become "normal" as you will miss a real error in the future. The possible errors are:

• Errors accessing folders: maybe a permissions issue. Check if the username is allowed to list and view the content of that folder
• Errors parsing: is the name of the folder meant to have the Case Reference in the correct format?
• Missing Case Ref: the Case Reference found in the folder name doesn't exist in ClariNet LS at all. Do you have a typo in the folder name, or a typo in the Case Reference in ClariNet LS? Do you need to create the Case in ClariNet LS?
• Empty Case Ref: unusual error, please contact ClearLife
• Download errors: whilst downloading the file by ftp for example, there can be an internet connection issue, or the user doesn't have the permission to download the file.
• Errors when adding: for example, the downloaded file is empty (0 bytes) and therefore not added to ClariNet LS. Check the file itself on the remote server.

Adding files to Cases

All the files are added in the Uncategorized Documents section, on the Case, Documents page. It would be very difficult to guess accurately which category to add documents in.

• Document types such as "Escrow Agreement" refer to the Case or "Policy Form" refers to the Policy. The Case and the Policy always exist, and there is only one of each. The match is easy.
• Document types such as "Hipaa Release" refer to an Insured. For Single Life Policy, the match is easy. But for Joint Life Policies, you have to decide manually which Insured to assign the Hipaa document to.
• Some Document types such as "Illustration" or "VOC" require that the relevant full object (an Illustration or a VOC) be created in ClariNet LS first. Once created, you can then assign the document to it. It is NOT possible to set a document type to Illustration and not have the Illustration entered first. See the "-- please select --"" dropdown on this screenshot, which would result in an error as this Case has no existing Illustrations to assign this document to. Once you click SAVE on the Documents page, these assigned Documents will move to the relevant section of the page:

To make this process easier, when you create or edit an Illustration for example, the Add a Document popup allows you to pick a document from the Uncategorized Category instead of uploading it from your disk:

Auto Import Details

Auto Import works in two stages: the Listing and the Import.

1. The Listing Process

• It lists the files recursively on the remote server, the first few lines of the log file.

It tells you how many files and folders there are, and errors:

2016-07-13 17:02:07.667 - Listing Start Time is 2016-07-13 17:02:07 UTC
2016-07-13 17:04:30.680 - Done listing
2016-07-13 17:04:30.680 - ------------------------------------
2016-07-13 17:04:30.683 - | Files found= 246 #
2016-07-13 17:04:30.683 - | Number of folders= 8 #
2016-07-13 17:04:30.683 - | Errors accessing folders= 0 #
2016-07-13 17:04:30.683 - ------------------------------------

• It then extracts the case reference for each file. Generally, the folder name contains the Case Reference.

2016-07-13 17:04:30.683 - Now extracting information (Case Reference, etc.) from filenames
2016-07-13 17:04:36.877 - Done extracting information
2016-07-13 17:04:37.630 - ------------------------------------
2016-07-13 17:04:38.117 - | Parsed files= 246 #
2016-07-13 17:04:38.677 - | Errors parsing= 0 #
2016-07-13 17:04:39.140 - ------------------------------------

• Then it checks each of these extracted Case Reference actually exists in your ClariNet LS client account.

2016-07-13 17:04:41.163 - Now checking Case References exist
2016-07-13 17:05:13.827 - Done checking Case References exist
2016-07-13 17:05:14.767 - ------------------------------------
2016-07-13 17:05:15.200 - | Missing Case Ref= 66 #
2016-07-13 17:05:15.517 - | Empty Case Ref= 0 #
2016-07-13 17:05:16.053 - ------------------------------------

• Finally, it generates the actual import task (a bunch of xml data), which will download each file.

In this example, it says 66 missing case ref. Open the detailed Log File which contains a lot more information than the summary.

Search for the word “ERROR”. You will find lines similar to this one:

2016-07-13 17:04:52.150 - ERROR. Case Reference 'SMITH-1234' does not exist in ClariNet. Ignoring file '/SMITH-1234/Issued Policy - Smith B. Jack.pdf'

To fix the error, search ClariNet LS for case reference “SMITH-1234”, you’ll probably find a mismatch, and correct either the Case Reference in ClariNet LS or the folder name in the remote server.

2. The Import Process

• It checks the XML file generated by the Listing Process, and downloads all the documents that don't already exist for the given Case Reference.

2016-07-14 10:27:50.773 - Case Ref 'SMITH-1234', downloaded new file '/SMITH-1234/ccagold_06152016_smith 6111234 change of beneficiary.pdf' (45786 bytes)
2016-07-14 10:27:53.723 - Case Ref 'SMITH-1234', downloaded new file '/SMITH-1234/ccagold_06152016_smith 6111234 change of ownership.pdf' (49591 bytes)
2016-07-14 10:27:53.727 - Done downloading
2016-07-14 10:27:53.727 - ------------------------------------
2016-07-14 10:27:53.727 - | Files downloaded= 179 #
2016-07-14 10:27:53.730 - | Ignored files (already exist)= 0 #
2016-07-14 10:27:53.730 - | Download errors= 1 #
2016-07-14 10:27:53.730 - ------------------------------------

• It then adds each file to the Case Documents, in the Uncategorized category. Note that when a file is empty (0 bytes) it is not added, as shown in the example below:

2016-07-14 10:27:53.730 - Now adding files to Cases
2016-07-14 10:27:53.847 - ERROR, DOCUMENT NOT ADDED: Document Data is empty, not adding to Case 'SMITH-1234', filename=/SMITH-1234/10-12-14 Smith Contract Request Form.docx.
2016-07-14 10:27:54.337 - Document added to Case 'SMITH-1234', filename=/SMITH-1234/SMITH, B - Illustration - 611234 - 08-20-2014.pdf.
2016-07-14 10:27:54.347 - Document added to Case 'SMITH-1234', filename=/SMITH-1234/Smith, B - 61234 - 2015 Annual Stmt.pdf.
2016-07-14 10:27:54.353 - Document added to Case 'SMITH-1234', filename=/SMITH-1234/Smith, B - 61234 - COB Confirm - WFNA.pdf.
2016-07-14 10:27:56.580 - Done adding
2016-07-14 10:27:56.580 - ------------------------------------2016-07-14 10:27:56.580 - | Files added= 179 #
2016-07-14 10:27:56.583 - | Errors when adding= 1 #
2016-07-14 10:27:56.583 - | Affected Cases= 6 #
2016-07-14 10:27:56.583 - ------------------------------------

• Finally, it reschedules itself for the next run (typically, the next day) and sends an email notification containing the Summary Log to the list of emails in the Notification Emails column.

File Transfer Details