Natural Language Understanding

Purpose: This is a feature to ask questions of a client or applicant and interpret natural language responses. This allows for asking the questions via SMS and Voice in addition to the standard web interface. 

Cost: $6,250 one time costs to enable and then $50/month. This feature is currently in Beta. 

This operates by having a Guided Navigation dialogue control the questions asked. That dialogue is then linked to Twilio to send the appropriate text messages and interpret the responses. 

NLU requires the SMS Module to operate, with a second phone number in Twilio. Sending NLU requests on existing cases requires the Workflows module. 

Use Cases

There are two major use cases for this feature. The first is to capture a basic intake via voice or SMS services.

The second (and more typical) is to send out a survey to a client via SMS after the case is closed. Rather than an External Form link which redirects to a LegalServer website, this asks the questions directly in the content of the SMS messages.

Configuration

Creating your Guided Navigation Dialogue

NLU operates through a Guided Navigation dialogue. Please note that not all blocks are supported, Custom Lookups are not supported, and only some lookups are currently supported. 

Field types supported by the NLU:

• Dates
• Numbers (only as numbers though)
• Booleans
• Text Fields
• Certain System Lookups:
  • Gender
  • How Referred
  • Troolean
  • Citizenship Status
  • Language

For dates, LegalServer uses the "Duckling" tool. Their documentation provides a wealth of examples that can be understood. 

For lookups, LegalServer first tries to match the hard coded responses in the lookup. If nothing matches, then it tries to classify the text to values that match either the lookup or the LSIndex value for the lookup. 

Additionally, any field marked as Confidential is not available in the NLU features. SMS is inherently insecure, so fields with sensitive information should not be included or requested. 

When creating your Guided Navigation Dialogue, there is a new field on each element you add to the segments to indicate if it should be included in the available Channels: 

The channels are whether this element is included in the given NLU channel. Web is the standard User Interface. SMS and Voice are for the NLU channels themselves. 

Best practice is to include a concluding instruction thanking the client for participating and indicating that the process is complete. 

Creating your Process

When adding the Dialogue to a form, it needs to be added to an External form (Intake or Auxiliary as desired). That form then needs to be added to an External process. Once the process is created, be sure to set the process as available via the Guided Navigation API.

An External Intake is needed for a new applicant. An External Auxiliary process is needed to reach out to an existing client. 

Connecting your Process to Twilio

Each type of NLU process (Intake and Auxiliary/Workflows) needs to be connected to a separate Twilio phone number. This can happen on the SMS & Telephony Admin page. Both these processes and the standard SMS integration all use different webhooks with LegalServer. If no unique phone number is established, then the standard Twilio Phone Number will be used. If the webhook identified in Twilio isn't set appropriately, then this will not work.

To get the webhook url to link to Twilio, you'll need to go to Admin -> NLU Intake Settings. Once you Select the External Aux process, you will get specific URLs to use with Twilio. 

Each process has its own default link, although the specified process is a backup only. Note that NLU Intake, NLU Auxiliary, and Standard LS SMS all use different webhooks. Only one phone number needs to be connected to each endpoint in Twilio. The NLU links include a number of default parameters ?channel=sms&pid=12354&h=2345 . These are defaults in case something is missing. They will be ignored if a different process was initiated by workflows.

Then Login to Twilio and Manage your phone numbers. When you find the phone number you want to associate with this particular process, you'll paste the new url to process into the settings:

If you are connecting voice processing, it should be via the Voice section. 

Triggering the NLU Process from a Workflow

When setting up the NLU Process to trigger from a Workflow, you can set the triggers for the workflow as standard. The key aspect for the NLU Process is to select an Action Step of Send NLU Process. An example follows: 

Workflows triggering on Time Since Event can be configured to have a minimum start time for the day to prevent the client from being woken up by the NLU messages. 

Workflow Triggering Settings:

There are four settings to configure when triggering NLU from a workflow.

• Custom Expiration Window allows you to change the expiration of the NLU workflow for the client to respond. The default is 7 days to finish the interaction. Full day increments can be set here.
• Wait for Prior Active/Queued NLU Processes Before Launching checks whether there is an existing process for the phone number already in the queue and then waits for that NLU Process to finish before this Process will trigger.
• Delay Launch After Prior Active NLU Dialogue Finishes or Expires sets a window between the completion of the prior process and the trigger of the new process.
• Send within Specific Time allows an additional way to trigger on the timing of the text messages. This is especially relevant when not triggering on Time Since Event.

Administrative Details

Each destination number should only have one texting interaction active at a time. LegalServer by default allows for up to 7 days to finish the interaction. That setting can be changed on the backend by LegalServer in full day increments. See the Custom Expiration Window.

You can send the message "start over" to clear all states for the session. This is only for testing and should not be used by end users. This may only work on the Intake NLU interactions. 

Sending the text "go back" should cause an attempt to return to the previous interaction (the most recently asked field, for example). This may only work on the Intake NLU interactions.  

Your site will have NLU Logs available in the Admin tab. These allow you to see information about what was sent or said to the applicant/client and their response. If their response was not parsed properly, you have the option to correct the learning model being used. 

Testing Suggestion

When testing the workflow features, we recommend using a Field Value Change on a boolean or lookup to trigger the SMS messages to send. This way you can easily test the text message interaction. When putting it into production, you may want a much more complicated trigger, but something simple to test the SMS dialogue with is strongly recommended. 

Configuration Recommendations:

• Have a separate copy of the guided navigation dialogue for internal review than the copy being used for the client.
• Have a separate process for additional languages with different values for the field labels and use the filter in the Workflow options.
• In the initial segment, use the Guided Navigation Set a Field Value to set a custom field "NLU Dialogue Sent" to True. Add a filter for this field being null or false to your Workflow triggering logic so that you don't end up sending the dialogue twice to the same client if the case is reopened and later re-closed.
• In the initial segment, use the Guided Navigation Set a Field Value to set a custom field for the date the dialogue was sent.
• The initial segment needs to have a field that the client interacts with. It cannot just be an instruction ahead of an output expression.

Known Bugs: 

When testing your text interaction, if the questions come out of order, edit and re-save the guided navigation dialogue to ensure that all the questions have unique UUIDs. This is relevant to dialogues created before January 15, 2023 on Live Sites.