A Conversation with Mini Me

What if you could talk to yourself about everything you’ve ever written?

I’ve spent years capturing notes, strategies, and reflections across roles and projects.
Then I built a system that lets me have a real conversation with all of it – in my own voice.

Why I Built It

Like most strategists, I’ve accumulated years of thinking – buried in OneNote pages, PowerPoint decks, and half-finished ideas. The insights were there, but they weren’t accessible. Even OCR scans still retained the messiness of my scribbles and the lack of structure in early notes.

I wanted to know: what would happen if I could ask my past work a question – and it could answer back?

How It Works

The first stage in any project like this is working out the tech. There were a number of criteria that I used:

• Does it actually do the job! Sounds obvious, but I didn’t want to spend lots of time figuring out bits of technology then finding out that it didn’t cover what I needed
• Price. My target was sub-$10 per month. I tried doing it for $0 per month but the difference in effort and quality was enormous, for just the price of a burger.
• My prior knowledge. There is some tech I know about and a lot that I don’t. I didn’t want to spend months learning something new, even if it could be argued it was more appropriate.
• Variety. It might have been easier to do everything in a single unified tech stack for example Microsoft, but that would make the project less interesting for me.

Given the above, this is what I went with:

Process at High Level

Step 1 – Collect all the Data

By far the most interesting, laborious but useful part of the process. I have around 14 years worth of information about all of the work I have done. With hindsight I would have had all of that formatted in the same way and well structured for AI ingestion.

That was absolutely not the case:

A lot of the work is the preprocessing often unstructured and poorly scribbled information. here are just a few of my data sources as well as lots of blog posts, Word documents, PowerPoint documents; really, everything.

• PowerPoint files
  These were manually selected and hardcoded into the script — a reasonable tradeoff given how few I needed to track.
• RSS Feeds
  • My blog at bjrees.com
  • A few curated industry insight feeds
• OneNote Notebooks, such as:
  • Project documentation (e.g. Skynet, The Oracle)
  • Notes from a Cambridge Judge Business School programme
  • Third-party and personal research logs
• iCloud Backups
  These contained archived slide decks and supporting materials.

Step 2 – Index

Once there is some structure to data you can start the next stage of building a database. I used Pinecone, a vector database, perfectly suitable for the task.

Step 3 – build an online interface where I couldn’t ask questions

Nothing fancy, but I wanted an easy way where I could ask questions without having to use the command line, firing up services or anything. For this I used Google Cloud Run extensively.

Step 4 – Voice

This stage was certainly unnecessary. But as mentioned part of this is about learning and growing your knowledge. I wanted to see if there was a way of using a cloud service to speak like me (a “Mini me!”). The following voice recordings are not me. they are from ResembleAI, and cost me around $2 to make. The one word it struggled with was BJREES sadly.

What Happened?

The conversation surprised me. what was particularly impressive and I think the most important part of the project is that it pulled out ideas from previous years into coherent answers. We all have a recency bias in our memories personally I’m a long way from having total recall. The world of marketing has evolved significantly but there are certain core concepts that have remained. A simple example – understanding your customers was the main task of marketing from the first project I did to the most recent.

Example exchanges:

Real me: “What’s the biggest barrier to marketing and product alignment?”
Mini me: “In your past notes you wrote that alignment isn’t the real issue — it’s trust in shared metrics.”

Real me: “What motivates you to keep building?”
Mini me: “You wrote once: progress feels meaningful when it changes how people work, not just what they do.”

The key point here is that though I remember writing this many years ago, I certainly didn’t remember the detail and I certainly didn’t remember the connections with other pieces of work that I did.

Why It Matters

it was enjoyable doing this work but crucially this wasn’t about creating a gimmick. It’s a glimpse of what happens when brand, knowledge, and AI converge.

Imagine your company being able to talk to its accumulated knowledge:

• Every customer insight.
• Every proposal, presentation, and lesson learned.
• All accessible through natural conversation.

I see this as a future direction for Brand intelligence – providing more to your customers to help them do their jobs well.

What I have now heard from a couple of recent clients, is the importance of actually tracking progress in AI search. As with any parts of marketing you want to creating great messaging material, content and so on; but you also need to know whether that effort is actually working or not.

I use a simple spreadsheet for doing this, attached below:

To use this spreadsheet, just manually work through each of the columns below, for each of the queries where you think you should appear *:

Query
→ Type the exact search query or question you used (e.g., “What is Generative Engine Optimization?”).

Date
→ Enter the date you performed the search (format: DD/MM/YYYY).

Platform (Google / Perplexity / ChatGPT etc.)
→ Note which platform or search engine you used (e.g., Google, Perplexity, ChatGPT).

Mode (Normal / Incognito / VPN)
→ Indicate how you accessed the platform (e.g., Incognito, Signed-in, VPN).

Category (SEO / AEO / GEO)
→ Choose which category the result fits into:

• SEO = Traditional search results
• AEO = AI or answer-engine snippet
• GEO = Full generative or LLM-based output

Result Source (Regular / AI Overview / AI Mode)
→ Identify where the result appeared:

• Regular Search = Standard search results
• AI Overview = Google’s AI summary panel
• AI Mode = Full generative output replacing the usual page

Result Type (Direct Link / Snippet / Generative Synthesis)
→ Note how your content appeared:

• Direct Link = SEO link in results
• Snippet = Text summary or mention
• Generative Synthesis = LLM or AI-generated paragraph referencing your content

Did you appear? (Yes / No)
→ Simply record whether your site, name, or content appeared.

How did it appear? (Quoted / Mentioned in Narrative)
→ Describe how you appeared:

• Quoted = Direct quotation or citation
• Mentioned in Narrative = Indirect or paraphrased mention

Screenshot File
→ Add the filename of your saved screenshot (e.g., Screenshots/7.png).

* Or, for help getting this set up for yourself, give me a shout on Contact Us to find out more!

When building search tools, intelligent assistants, or AI-driven Q&A systems, one of the most foundational decisions you’ll make is how to retrieve relevant content. Most systems historically use keyword-based search—great for basic use cases, but easily confused by natural language or synonyms.

That’s where embedding-based retrieval comes in.

In this guide, I’ll break down:

• The difference between keyword and embedding-based retrieval
• Real-world pros and cons
• A step-by-step implementation using OpenAI and Pinecone
• An alternative local setup using Chroma

Keyword Search vs. Embedding Search

Keyword-Based Retrieval

How it works:
Searches for exact matches between your query and stored content. Works best when both use the same words.

Example:
Query: "What is vector search?"
Returns docs with the exact phrase "vector search".

Pros:

• Very fast and low-resource
• Easy to explain why a match was returned
• Great for structured and exact-match data

Cons:

• Doesn’t understand synonyms or phrasing differences
• Fails if the words aren’t an exact match

Embedding-Based Retrieval (Semantic Search)

How it works:
Both queries and documents are converted into dense vectors using machine learning models (like OpenAI’s text-embedding-ada-002). The system compares their semantic similarity, not just their words.

Example:
Query: "How does semantic search work?"
Returns docs about “meaning-based search” even if the words are different.

Pros:

• Understands intent, not just keywords
• Great for unstructured content and natural queries
• Can surface more relevant results even if phrasing is varied

Cons:

• More computationally intensive
• Results are harder to explain (based on vector math)
• Requires pre-trained models and a vector database

Feature Comparison Table

Setting Up Embedding-Based Retrieval

Let’s build a basic semantic search system using:

• OpenAI (text-embedding-ada-002)
• Pinecone (hosted vector DB)
• Chroma (optional local alternative)

1. Choose Your Tools

Embedding model:
OpenAI’s text-embedding-ada-002 or a local Hugging Face model.

Vector database:
Cloud: Pinecone (scalable, managed)
Local: Chroma (open-source, lightweight)

2. Install Required Libraries

pip install openai pinecone-client chromadb

3. Set API Keys

export OPENAI_API_KEY="your-openai-key"
export PINECONE_API_KEY="your-pinecone-key"

In Python:

import openai
openai.api_key = "your-openai-key"

4. Generate Embeddings

def get_embedding(text):
    response = openai.Embedding.create(
        input=text,
        model="text-embedding-ada-002"
    )
    return response['data'][0]['embedding']

documents = [
    {"id": "1", "text": "This is an introduction to embedding-based search."},
    {"id": "2", "text": "Embedding-based retrieval finds similar meanings."},
]

for doc in documents:
    doc['embedding'] = get_embedding(doc["text"])

5. Store in Pinecone

import pinecone

pinecone.init(api_key="your-pinecone-key", environment="us-east-1")

index_name = "embeddings-index"
pinecone.create_index(index_name, dimension=1536)

index = pinecone.Index(index_name)

to_upsert = [(doc['id'], doc['embedding'], {"text": doc["text"]}) for doc in documents]
index.upsert(vectors=to_upsert)

6. Perform a Semantic Search

query = "How does semantic search work?"
query_embedding = get_embedding(query)

results = index.query(query_embedding, top_k=5, include_metadata=True)

for match in results["matches"]:
    print(f"ID: {match['id']} | Score: {match['score']}")
    print(f"Text: {match['metadata']['text']}\n")

Optional: Use Chroma for Local Embedding Search

import chromadb

client = chromadb.Client()
collection = client.create_collection("documents")

for doc in documents:
    collection.add(
        documents=[doc["text"]],
        embeddings=[doc["embedding"]],
        ids=[doc["id"]]
    )

query_result = collection.query(query_texts=["How does embedding retrieval work?"], n_results=5)
print(query_result)

Evaluate the Results

Once you’re set up:

• Check result relevance
• Tune your top_k or switch models if needed
• Add keyword filtering for hybrid search

You now have a foundation for building:

• Intelligent assistants
• Internal knowledge base search
• Chatbots that retrieve based on meaning

What’s Next?

You can scale this up to thousands or millions of documents. Consider:

• Crawling blogs, docs, or Notion pages
• Combining embeddings with filters or metadata
• Using hybrid keyword + embedding pipelines for speed and precision

This is the part where all the content sources came together into a centralized system I could actually interact with.

This post is a cleaned-up record of what I built, what worked, what didn’t, and what I planned next. If you’ve ever tried to unify fragmented notes, decks, blogs, and structured documents into a searchable system, this might resonate 🙂

What I Built

There were two main components at the heart of the system:

1. Batch Processing Script
   PopulateChatSystemDataRepository.py — this was run manually to gather and format all source data into a single repository. My plan was to automate it later.
2. Continuous Scanner
   A lightweight background service monitored for new blog posts and updates.

At that point, the batch script did the heavy lifting, though I intended to shift it onto Google Cloud Run to handle scale.

Where the Data Lived

The sources I processed included:

• PowerPoint files
  These were manually selected and hardcoded into the script — a reasonable tradeoff given how few I needed to track.
• RSS Feeds
  • My blog at bjrees.com
  • A few curated industry insight feeds
• OneNote Notebooks, such as:
  • Project documentation (e.g. Skynet, The Oracle)
  • Notes from a Cambridge Judge Business School programme
  • Third-party and personal research logs
• iCloud Backups
  These contained archived slide decks and supporting materials.

All of this data was funneled into a staging area for eventual vector embedding and retrieval.

Microsoft Graph API + OneNote

To pull content from OneNote, I used the Microsoft Graph API. First, I installed the required libraries:

pip install msal requests

• msal handled authentication via Azure Active Directory
• requests allowed me to interact with the Graph API endpoints

Once I authenticated, I could enumerate and query notebooks like this:

python ExtractNotes.py

After logging in via a Microsoft-generated URL, I could successfully extract content from all the notebooks I needed.

Licensing Curveballs

At the time, I hit a snag: my Microsoft 365 Family plan didn’t include SharePoint Online, which was required to query OneNote via the Graph API.

I weighed my options:

1. Pay for a Business Standard plan (~£9.40/month)
2. Try and use my home license in some way, even thought it didn’t seem to have what I needed for OneNote

I went with option 2, supported by a one-month free trial of Microsoft Business Basic to help validate the approach.

Google Sheets as the Backbone

The ingestion script used a JSON keyfile to interact with Google Sheets. It opened the sheet like this:

client.open_by_key(sheet_id).sheet1

Sheets acted as a live database — but I ran into 429 rate-limit errors, especially when repeatedly reading the same files. To solve this, I built a basic checkpointing system so the script would:

• Cache previously processed records
• Avoid re-downloading the same content every time
• Track progress and only fetch new entries on each run

The GitHub Reset

After a short break from the project, I realized the codebase had grown too complex. I had introduced a lot of logic to deal with throttling and retries, but it made everything harder to understand.

So I rolled back to a much earlier commit and started again from a simpler foundation.

It was the right move.

What Came Next

Here’s what I tackled after that cleanup:

• Migrated the whole project to an old home laptop
• Simplified the ingestion pipeline
• Ensured each run processed only new data, not the full archive
• Finalized access and querying via Microsoft Graph API for OneNote and SharePoint content

Reflections

Skynet began as a chatbot experiment, but evolved into something bigger — a contextual knowledge system that drew from years of notes, presentations, and personal writing.

Stage 2 was about turning chaos into structure. The next phase was even more exciting: embeddings, retrieval, and building a system that could answer real questions, grounded in my own work.

Read Stage 1 if you missed the start.