2. QNA Maker .AI what's under the hood?

Category: meta

After deploying QNA Maker .AI I looked how it is deployed and what is going under the hood.

What I found was a Windows machine with combination of .NET and Python NLP applications. Wandering in command shell I stumble upon the configuration file claiming that return value is a weighted sum of 3 similarity scores TF-IDF (0.1), Wordnet (0.5) and AzureSearch (0.4).

Then I run a simple test from my machine to QNA Maker.Ai engine deployed in Azure US West 2 (~7 miles as raven flies):

$ cat rate.sh
#/bin/sh

for i in {1..100}; do
      line=$(shuf questions.txt | head -n 1)
      curl --silent -X POST https://qna-maker-test-on-basic-machine.azurewebsites.net/qnamaker/knowledgebases/.../generateAnswer \
      -H "Authorization: EndpointKey ..." -H "Content-type: application/json" -d "{'question':'$line'}" > /dev/null
done

$ time rate.sh
real	0m59.050s
...

and the same in “US West” region (S1 plan + Basic Azure Cognitive search cluster):

real	1m4.411s
...

Same test run from that Windows command shell of qnamaker.ai (conf. above, q100 contains 100 questions):

$more test.bat
@ECHO OFF
echo %time%
for /F "tokens=*" %%i in (q100.txt) do @curl --silent -X POST https://qnamakerai-app.azurewebsites.net/qnamaker/knowledgebases/.../generateAnswer `
                                             -H "Authorization: EndpointKey ..." -H "Content-type: application/json" -d "{'question': '%%i'}" >ttt
echo %time%

$ test.bat
start 16:18:12.67
finish 16:19:32.54

1:19.85 - probably depends on the question set - and this one turn out to be an uncomfortable set for the algorithms.

So, considering all above - can we do better? And if yes - how much better can we do?