Module 4: Chatbot #1 - Scraping Assistant

You have your Scrapy documentation. Now let’s build your first Zapier chatbot - a Scrapy tutor that helps you build scrapers.

Why Zapier Chatbots?

Zapier offers a free chatbot tier that lets you:

• Upload markdown files as knowledge base
• Set custom directives
• Embed on your website or use via Zapier

The catch: free tier = limited bots. We’ll cover rotation strategy in Module 9.

Step 1: Create Your Zapier Chatbot

1. Go to zapier.com/ai/chatbot
2. Sign up / log in (free account)
3. Click “Create Chatbot”
4. Name it: Scrapy Tutor

Step 2: Upload Knowledge Base

1. In chatbot settings, find “Knowledge Base”
2. Upload your scraped Scrapy markdown files
3. Wait for processing (may take a few minutes)

Tip: Start with 5-10 files max. More files = more tokens = higher costs.

Step 3: Write Your TOML Directive

The directive is your chatbot’s personality and instructions. Create this in Notepad:

# ===========================================
# SCRAPY TUTOR - Chatbot Directive
# ===========================================

[user_profile]
# Tell the bot about the user
technical_ability = "beginner"
operating_system = "Windows 11"
tools_available = ["VSCode", "Python 3.11", "GitBash"]
goal = "Learn web scraping to build personal AI knowledge bases"

[chatbot_directive]
# Who is the bot?
role = "Scrapy tutor and code assistant"
personality = "patient, encouraging, practical"
tone = "friendly but professional"

# What should it do?
primary_task = "Help users build Scrapy spiders to scrape web content"
response_style = "concise, with code examples when helpful"

# What to avoid?
do_not = [
    "don't repeat instructions unnecessarily",
    "don't use jargon without explaining",
    "don't provide overly complex solutions to beginners"
]

# Where can the bot be creative?
uncertainty_areas = [
    "creative problem solving when standard approaches fail",
    "suggesting alternative selectors when standard CSS/XPath doesn't work",
    "troubleshooting unusual website structures"
]

# Special instructions
format_preference = "Use markdown formatting in responses"
code_blocks = "Always provide code in triple backtick blocks"

Step 4: Paste Directive to Zapier

1. In chatbot settings, find “Directive” or “Instructions”
2. Paste your TOML (Zapier will parse it as text)
3. Save

Step 5: Test Your Chatbot

Try these prompts:

“How do I create a new Scrapy project?”

“I want to scrape articles from community.zapier.com. Where do I start?”

“What’s the difference between CSS selectors and XPath?”

Refining Your Directive

If responses aren’t helpful:

1. Too complex? Lower technical_ability or add more do_not items
2. Too simple? Raise technical_ability
3. Not staying on topic? Add more specific primary_task details
4. Repeating itself? Add to do_not list

Example Directive Iteration

Before (v1):

role = "Help with Scrapy"

After (v2):

role = "Scrapy tutor for beginners"
do_not = ["don't explain advanced topics unless asked"]

Key Takeaways

1. Zapier free tier allows chatbot creation with knowledge base
2. TOML directive personalizes the bot for your skill level
3. Test and refine - Iterate on your directive
4. Start small - 5-10 KB files first

Next Steps

Now let’s build Chatbot #2 - your Zapier Specialist. Continue to Module 5: Chatbot #2 - Zapier Specialist