Natural text interpreter

I have been confronted with following problem: 2000 pages of textual test-procedures, nobody want to have anything to do with it. I will shortly present an approach to develop a light-weight and tolerant interpreter, and a second one with the usage of AI.

Concept

Each file is parsed individually, row by row with the help of a class Row. From the beginning of the row, the Row tries to match any kind of IExpression as long as there is some text left. Each successful match produces a new IInterpretedElement, then the parsing continues with the remaining text of the row.

You will find here an implementation example which demonstrate the parsing of a single row. It defines 3 different grammar expressions with a Regex pattern: "^Select the menu(.*)$", "^and(.*)$" and "click the button(.*)$".

The string "Select the menu and click the button" is parsed as follow:

The parsing ends after the 3rd step since there is no remaining text. In the implementation we have also introduced a class "ExpressionOr" to collect all the expressions of the grammar which can be permuted in a list. Since the first match is successful (potential additional match are not checked), we have to pay attention to the order.

LL(k) considerations

The notation LL(k) is typically used to qualify a grammar and its complexity, meaning by this that it can be analysed from the left to the right. Since we are using RegEx it's hard to say how we are fitting to the definition but since our grammar it is determistic (it can be read in one flow from the left to the right) we may say it is a LL(1) grammar.

Shift into second gear: AI

Describing the complete grammar may become tedious, let's check if the task can be easily done by ChatGPT. For this we will use Azure OpenAI service, and experiment a completion model.

Here the approach is different: in a first step we "teach" the AI what is the correct answer of a question and let it run standalone with the row analysis. The answer is useful to define the expected format:

Chat gpt's answer is a kind of Json format, which can be easily used in a program: { Action: 'Select', Object: 'menu' }, { Action: 'Click', Object: 'button' }.

You will find the exemple behind this link: you will need the URL of the server and also the Azure key.

Conclusion

As we could see it is possible to write a text interpreter with only a few lines of code. With the help of RegEx we do not need to care about each token (letter) individually. On the other hand, the completion model is even easier to use and produces spectacular results.