In the age of AI and machine learning, ensuring that documents are formatted in a way that is easy for AI models to understand and process is crucial. Proper formatting enhances the accuracy of AI-generated insights, especially when the content includes complex elements like images, tables, and unstructured text.

Why Is It Needed?

AI models, especially large language models (LLMs), excel at processing structured data but struggle with complex, unorganized content. Without clear headings, bullet points, or structured tables, an AI may fail to extract the necessary insights from a document. This is where document formatting becomes essential. Structured documents allow AI systems to understand the relationships between different parts of the content, improving both the accuracy of information extraction and AI-generated responses.

How to Achieve It

To make documents AI-friendly, the following steps can be taken:

Convert Images to Text: Use Optical Character Recognition (OCR) or image analysis tools to extract text from images and summarize content such as charts or screenshots.

Use Clear Headings and Subsections: Break the document into clear sections, with titles and bullet points to help the AI recognize and categorize the information.

Summarize Tables and Graphs: Convert tables and graphs into structured text or summaries so that the model can process and understand the data more easily.

Provide Contextual Instructions: When interacting with AI, include instructions for specific tasks, such as extracting text or summarizing content, to improve the response quality.

Here’s an example of how you can use Python code to automate parts of this process. The code below demonstrates how to convert an image to a data URL, and use an AI model (Azure OpenAI) to extract and structure the content.

python code

import os
import requests
import base64
from openai import AzureOpenAI
from mimetypes import guess_type
import re
import docx

# Initialize Azure OpenAI LLM
def initialize_llm():
try:
llm = AzureOpenAI(
api_key="your_api_key", # Replace with your Azure API key
api_version="your_api_version",
)
print("Azure OpenAI model initialized successfully.")
return llm
except Exception as e:
print(f"Error initializing LLM: {e}")
raise

def save_to_docx(reformatted_text, output_path):
doc = docx.Document()
doc.add_paragraph(reformatted_text)
doc.save(output_path)
print(f"Reformatted document saved as: {output_path}")

# Convert Image to Data URL
def local_image_to_data_url(image_path):
# Get mime type
mime_type, _ = guess_type(image_path)

if mime_type is None:
mime_type = 'application/octet-stream'

with open(image_path, "rb") as image_file:
base64_encoded_data = base64.b64encode(
image_file.read()).decode('utf-8')

return f"data:{mime_type};base64,{base64_encoded_data}"

# Main Execution
if __name__ == "__main__":

# Generate Token
llm = initialize_llm()
data_url = local_image_to_data_url('image_path')

# Send Request to Azure OpenAI
response = llm.chat.completions.create(
model="your_model_name"
messages=[{
"role": "system",
"content": "You are an AI helpful assistant."
}, {
"role": "user",
"content": [{
"type": "text",
"text": """
Please extract the text from the image, including any text found in tables, summary of nested images(includes screenshots of application , dashboards), and other elements. 
Reformat the extracted text into a structured format that makes it easier to parse and understand. 
The document should be divided into clearly defined sections, with text organized into:
- Titles/Headings
- Subsections or bullet points where appropriate
- Tables, graphs, or images should be summarized with structured details.
Ensure all the extracted content is clear and formatted for easy reading by a language model.
"""
}, { 
"type": "image_url",
"image_url": {
"url": data_url
}
}]
}],
max_tokens=4000,
temperature=0.7
)

if response.choices:
img_description = response.choices[0].message.content
print(img_description)
save_to_docx(img_description, 'your_doc_name')

Applications

1. Automating SOP Creation: By extracting information from images and text, and then structuring it, AI can generate Standard Operating Procedures (SOPs) that are easy to parse and understand.

2. Real-Time Querying: Once a document is formatted for AI parsing, users can ask the AI specific questions about the content. For example, querying about what happens when a user clicks on a particular item in an image or table.

Conclusion

Proper document formatting is key to making content easily readable by AI models. By converting images to text, using structured headings and bullet points, and summarizing complex elements like tables, documents become much easier for AI to analyze. This leads to better insights and more accurate responses when the AI is queried. The code presented here automates the process, making the task of document structuring both efficient and scalable, ensuring that your documents are AI-friendly and ready for real-time interactions.

Neha Vittal Annam