What I’m trying to do:
I made an app to use a large language model to take a paragraph as input and return 3-10 MCQ questions as output.
What I’ve tried and what’s not working:
In my python code I made a function that uses a prompt template and feeds it to the LLM along with the paragraph text and returns the response generated by the LLM.
I then made the UI in anvil and wrote the python code in anvil correctly while following a youtube video made by the anvil team.
The issue is that the output that is displayed in the website consists of the prompt and the paragraph text and then only the MCQ questions come. This is not how its supposed to be, its supposed to only give out the MCQ questions.
My response generation function code:
@anvil.server.callable
def generate_mcqs(paragraph):
prompt = f"""Your job is to take a given text and generate a minimum of 3 and maximum of 10 multiple choice questions.
The questions should be related to the text. The questions should be in the following format:
Question: ....
A: ....
B: ....
C: ....
D: ....
Answer: ....
The answer should contain only the correct option without any explanation or any other information.
The text is:
{paragraph}"""
inputs = tokenizer(prompt, return_tensors="pt").to('cuda')
outputs = model.generate(**inputs, max_new_tokens=512).to('cuda')
response = tokenizer.decode(outputs[0], skip_special_tokens=True)
return response
Please note that the variable response contains the MCQ questions and only that should be displayed.
Anvil code:
from ._anvil_designer import Form1Template
from anvil import *
import anvil.server
class Form1(Form1Template):
def __init__(self, **properties):
# Set Form properties and Data Bindings.
self.init_components(**properties)
# Any code you write here will run before the form opens.
def gen_response_click(self, **event_args):
"""This method is called when the button is clicked"""
model_output = anvil.server.call('generate_mcqs', self.paragraph.text)
if model_output:
self.response.visible = True
self.response.text = model_output
Here’s a screenshot of my output:
Note: this is not the complete output because it is too large to fit but i hope you get the point that i shouldn’t be getting the prompt and the paragraph that the user inputs, i should be getting the MCQs generated by the model only.