Fix: Added ai_business_insider as a regular folder with code files

This commit is contained in:
Madhu 2024-11-06 04:31:34 +05:30
parent ac5affaeba
commit 06447e73e7
32 changed files with 7429 additions and 1 deletions

@ -1 +0,0 @@
Subproject commit 869ccc8d115f37a708c157ec6824b14635dc2cd4

View file

@ -0,0 +1 @@
venv

Binary file not shown.

View file

@ -0,0 +1,117 @@
## AI Business Insider - A News Summarizer and Analyst - CrewAI
- Problem Statement: This is quite Personal to me, since my 1st year of Uni, I always wanted to be an Entreprenuer. Back then, I was constantly looking for a gap in the market, a problem that I can solve by constantly by reading news articles extensively, which was a very hard task. I have build this AI Agent News Analyzer that gives me Business Ideas regarding a particular topic. I have realised now that there's no perfect idea - atleast in the tech space. You have to build stuff and gain experience working with multiple people increasing your odds of finding a problem to work on rather than reading news.
This project is a News Engine that helps users research a news topic, fact-check information, summarize key findings, and analyze trends to help find their Potential Business Ideas for their own Company!. It leverages multiple AI agents to collect, verify, and analyze news, enabling users to extract insights and identify key opportunities across various news articles. The application uses [CrewAI](https://crewai.com/) to orchestrate these tasks and provide an interactive experience.
## Features
- **User Prompt**: Users can input a topic of interest for research.
- **News Collection**: The system gathers recent news articles based on the topic provided.
- **Fact Verification**: Key facts from collected news articles are verified using trusted tools.
- **Summary Generation**: Concise summaries of verified information are generated.
- **Trend Analysis**: The system identifies trends and patterns across the analyzed news stories, providing deeper insights for user's potential business ideas
## Architecture
This tool comprises four key agents:
### 1. News Collector
- **Task**: Collects recent news articles on the given topic.
- **Tools**:
- News APIs (like [NewsAPI](https://newsapi.org/))
### 2. Fact Checker
- **Task**: Verifies key facts from collected articles.
- **Tools**:
- Google Fact-Check Tools API
### 3. Summary Writer
- **Task**: Produces concise summaries of the verified information.
- **Tools**:
- Hugging Face's [BART](https://huggingface.co/facebook/bart-large-cnn) / GPT 4o (now)
### 4. Trend Analyzer
- **Task**: Analyzes trends and patterns across collected and summarized news stories.
- **Tools**:
- OpenAI's GPT 4o
## Project Flow
1. **Step 1**: User enters a topic in the input field on the frontend.
2. **Step 2**: The system fetches recent news articles related to the topic using the News Collector agent.
3. **Step 3**: The Fact Checker agent verifies the accuracy of key facts from the news articles.
4. **Step 4**: The Summary Writer agent generates a concise summary of the news.
5. **Step 5**: The Trend Analyzer agent examines the news for trends and patterns.
6. **Step 6**: Results are presented to the user, showing collected news, verified facts, a summary, and trend analysis.
## Technologies Used
- **Frontend**: Streamlit
- **Backend**:
- Python (with FastAPI for the API backend)
- **AI/ML Tools**:
- CrewAI for orchestrating AI agents
- Google Fact-Check Tools API
- OpenAI for trend analysis
- **Web Scraping**: Custom web scrapers and external APIs (News API)
## How to Run
1. Clone the repository:
```bash
git clone https://github.com/Madhuvod/AI-Business-Insider.git
cd AI-Business-Insider
```
2. Create and activate a virtual environment:
```bash
# For macOS/Linux
python -m venv venv
source venv/bin/activate
# For Windows
python -m venv venv
.\venv\Scripts\activate
```
3. Install the required packages:
```bash
pip install -r requirements.txt
```
(after doing this, sometimes a few packages might not be installed - so installed those 2-3 packages seprately, or reload the IDE if any import errors showing)
4. Create a new .env file and set up your environment variables:
```bash
# Get API keys from:
# - News API: https://newsapi.org/account
# - Google Fact Check: https://developers.google.com/fact-check/tools/api
# - OpenAI: https://platform.openai.com/api-keys
NEWS_API_KEY=your_news_api_key
GOOGLE_FACT_CHECK_KEY=your_google_fact_check_key
OPENAI_API_KEY=your_openai_api_key
```
5. Run the application (in two different terminals):
```bash
# Terminal 1 - Start the backend
python news_summarizer_analyzer/src/news_summarizer_analyzer/main.py
# Terminal 2 - Start the Streamlit frontend
streamlit run news_summarizer_analyzer/src/news_summarizer_analyzer/streamlit_app.py
```
## Development Setup
If you're contributing to the project:
```bash
# Install in development mode
pip install -e .
# Install additional development dependencies
pip install pytest python-dotenv
```
**TODO**: Thinking of using A voice agent for this tooo
![image](IMG_3530.heic)

View file

@ -0,0 +1,3 @@
.env
__pycache__/
env

View file

@ -0,0 +1,54 @@
# NewsSummarizerAnalyzer Crew
Welcome to the NewsSummarizerAnalyzer Crew project, powered by [crewAI](https://crewai.com). This template is designed to help you set up a multi-agent AI system with ease, leveraging the powerful and flexible framework provided by crewAI. Our goal is to enable your agents to collaborate effectively on complex tasks, maximizing their collective intelligence and capabilities.
## Installation
Ensure you have Python >=3.10 <=3.13 installed on your system. This project uses [Poetry](https://python-poetry.org/) for dependency management and package handling, offering a seamless setup and execution experience.
First, if you haven't already, install Poetry:
```bash
pip install poetry
```
Next, navigate to your project directory and install the dependencies:
1. First lock the dependencies and install them by using the CLI command:
```bash
crewai install
```
### Customizing
**Add your `OPENAI_API_KEY` into the `.env` file**
- Modify `src/news_summarizer_analyzer/config/agents.yaml` to define your agents
- Modify `src/news_summarizer_analyzer/config/tasks.yaml` to define your tasks
- Modify `src/news_summarizer_analyzer/crew.py` to add your own logic, tools and specific args
- Modify `src/news_summarizer_analyzer/main.py` to add custom inputs for your agents and tasks
## Running the Project
To kickstart your crew of AI agents and begin task execution, run this from the root folder of your project:
```bash
$ crewai run
```
This command initializes the news-summarizer-analyzer Crew, assembling the agents and assigning them tasks as defined in your configuration.
This example, unmodified, will run the create a `report.md` file with the output of a research on LLMs in the root folder.
## Understanding Your Crew
The news-summarizer-analyzer Crew is composed of multiple AI agents, each with unique roles, goals, and tools. These agents collaborate on a series of tasks, defined in `config/tasks.yaml`, leveraging their collective skills to achieve complex objectives. The `config/agents.yaml` file outlines the capabilities and configurations of each agent in your crew.
## Support
For support, questions, or feedback regarding the NewsSummarizerAnalyzer Crew or crewAI.
- Visit our [documentation](https://docs.crewai.com)
- Reach out to us through our [GitHub repository](https://github.com/joaomdmoura/crewai)
- [Join our Discord](https://discord.com/invite/X4JWnZnxPb)
- [Chat with our docs](https://chatg.pt/DWjSBZn)
Let's create wonders together with the power and simplicity of crewAI.

File diff suppressed because it is too large Load diff

View file

@ -0,0 +1,22 @@
[tool.poetry]
name = "news_summarizer_analyzer"
version = "0.1.0"
description = "news-summarizer-analyzer using crewAI"
authors = ["Your Name <you@example.com>"]
[tool.poetry.dependencies]
python = "^3.10"
tomli = "^2.0.1"
crewai = { extras = ["tools"], version = ">=0.70.1,<1.0.0" }
[tool.poetry.scripts]
news_summarizer_analyzer = "news_summarizer_analyzer.main:run"
run_crew = "news_summarizer_analyzer.main:run"
train = "news_summarizer_analyzer.main:train"
replay = "news_summarizer_analyzer.main:replay"
test = "news_summarizer_analyzer.main:test"
[build-system]
requires = ["poetry-core"]
build-backend = "poetry.core.masonry.api"

View file

@ -0,0 +1,87 @@
from crewai_tools import BaseTool
from typing import Dict, List, Type
from pydantic import Field
import requests
import os
from pydantic import BaseModel, Field
class Article(BaseModel):
title: str
url: str
snippet: str
class FactCheckerInput(BaseModel):
articles: List[Article] = Field(
...,
description="List of articles to fact-check",
example=[{
"title": "Sample Article",
"url": "https://example.com",
"snippet": "Sample article content"
}]
)
class FactCheckerTool(BaseTool):
name: str = "Fact Checker"
description: str = "Verifies the factual accuracy of news articles using Google Fact Check API."
args_schema: Type[BaseModel] = FactCheckerInput
api_key: str = Field(default_factory=lambda: os.getenv("GOOGLE_FACT_CHECK_API_KEY", ""), description="API key for Google Fact Check API")
base_url: str = "https://factchecktools.googleapis.com/v1alpha1/claims:search"
def _run(self, articles: List[Article]) -> List[str]:
"""
Verifies the factual accuracy of news articles.
Args:
articles: A list of Article objects to fact-check.
Returns:
A list of titles of articles that are considered factual.
"""
# Ensuring articles is a list of Article objects
if isinstance(articles, str):
raise ValueError("Expected a list of Article objects, got a string")
if not isinstance(articles, list):
raise ValueError("Expected a list of Article objects")
factual_articles = []
for article in articles:
query = f"{article.title} {article.snippet}"
params = {
'key': self.api_key,
'query': query
}
try:
response = requests.get(self.base_url, params=params)
response.raise_for_status()
fact_checks = response.json().get('claims', [])
# If no fact checks found, consider the article potentially factual
if not fact_checks:
factual_articles.append(article.title)
continue
# Check each claim review
for check in fact_checks:
claim_reviews = check.get('claimReview', [])
if not claim_reviews:
continue
rating = claim_reviews[0].get('textualRating', '').lower()
# Consider article factual if no clear false/fake indicators
if not any(indicator in rating for indicator in ['false', 'fake', 'pants on fire', 'incorrect']):
factual_articles.append(article.title)
break
except requests.RequestException as e:
print(f"Error checking facts for article '{article.title}': {str(e)}")
# In case of API error, we'll include the article but log the error
factual_articles.append(article.title)
return factual_articles
async def _arun(self, articles: List[Dict[str, str]]) -> List[str]:
"""Async implementation of the tool"""
return self._run(articles)

View file

@ -0,0 +1,50 @@
from crewai_tools import BaseTool
import requests
from typing import List, Dict, Literal
from pydantic import Field
import os
class NewsAPITool(BaseTool):
name: str = "News API"
description: str = (
"Fetches the latest news articles based on a given query using the News API."
)
api_key: str = Field(default_factory=lambda: os.getenv("NEWS_API_KEY", ""), description="API key for the News API")
base_url: Literal["https://newsapi.org/v2/everything"] = "https://newsapi.org/v2/everything"
def __post_init__(self):
if not self.api_key:
raise ValueError("API key for News API is not set in the environment variables.")
def _run(self, query: str) -> List[Dict[str, str]]:
"""Fetches news articles based on a query"""
params = {
'q': query,
'apiKey': self.api_key
}
try:
response = requests.get(self.base_url, params=params)
response.raise_for_status()
articles = response.json().get('articles', [])
return [
{
"title": article["title"],
"url": article["url"],
"source": article["source"]["name"],
}
for article in articles[:5]
]
except requests.RequestException as e:
raise Exception(f"Error fetching news articles: {str(e)}")
async def _arun(self, query: str) -> List[Dict[str, str]]:
"""Async implementation of the tool"""
return self._run(query)
# def news_collector():
# api_key = os.getenv("NEWS_API_KEY")
# if not api_key:
# raise ValueError("API key for News API is not set in the environment variables.")
# tools = [NewsAPITool(api_key=api_key)]
# # ... rest of the code ...

View file

@ -0,0 +1,54 @@
from crewai_tools import BaseTool
from typing import Dict, List, Any, Type
from pydantic import Field, BaseModel
from openai import OpenAI
import os
class Article(BaseModel):
name: str
class SummaryWriterInput(BaseModel):
articles: str # Changed from List[Article] to str because of an errorr
class SummaryWriterTool(BaseTool):
name: str = "Summary Writer"
description: str = "Generates concise summaries of news articles."
api_key: str = Field(default_factory=lambda: os.getenv("OPENAI_API_KEY", ""), description="API key for OpenAI")
args_schema: Type[BaseModel] = SummaryWriterInput
def _run(self, articles: str) -> Dict[str, Any]: # Change parameter type to str
"""
Generate a summary for the given articles using OpenAI API.
:param articles: A string containing article names.
:return: A dictionary with summary and metadata.
"""
client = OpenAI(api_key=self.api_key)
# Use the articles string directly
combined_content = articles
response = client.chat.completions.create(
model="gpt-3.5-turbo",
messages=[
{"role": "system", "content": "You are a helpful assistant."},
{"role": "user", "content": f"Summarize the following article titles and provide key points: {combined_content}"}
],
max_tokens=300
)
summary_text = response.choices[0].message.content.strip()
# Split the summary into summary and key points
parts = summary_text.split("Key points:")
summary = parts[0].strip()
key_points = parts[1].strip().split("\n") if len(parts) > 1 else []
return {
"title": "Summary of articles",
"summary": summary,
"key_points": key_points,
"sources": combined_content.split(", ") # Assuming articles are comma-separated
}
async def _arun(self, articles: str) -> Dict[str, Any]: # Change parameter type to str
return self._run(articles)

View file

@ -0,0 +1,163 @@
from crewai_tools import BaseTool
from typing import Dict, List, Union, Optional
from openai import OpenAI
import os
from pydantic import Field, BaseModel
import json
import logging
# Set up logging
logging.basicConfig(level=logging.INFO)
logger = logging.getLogger(__name__)
class InputValidationError(Exception):
"""Custom exception for input validation errors"""
pass
class APIError(Exception):
"""Custom exception for API-related errors"""
pass
class TrendAnalyzerTool(BaseTool):
name: str = "Trend Analyzer"
description: str = "Analyzes trends across multiple news article summaries and suggests potential outcomes and business ideas."
api_key: str = Field(default_factory=lambda: os.getenv("OPENAI_API_KEY", ""), description="API key for OpenAI")
def validate_input(self, data: Dict) -> None:
"""
Validates the input data structure.
Args:
data: Dictionary containing the input data
Raises:
InputValidationError: If the input data is invalid
"""
if not isinstance(data, dict):
raise InputValidationError("Input must be a dictionary")
if "key_points" not in data and "summary" not in data:
raise InputValidationError("Input must contain either 'key_points' or 'summary'")
if "key_points" in data and not isinstance(data["key_points"], list):
raise InputValidationError("key_points must be a list")
def _run(self, input_data: Union[str, Dict]) -> Dict[str, list]:
"""
Analyze summaries to identify trends and suggest outcomes and business ideas.
Args:
input_data: A dictionary containing structured news summaries
Returns:
Dict[str, list]: A dictionary with trends, outcomes, and business ideas
"""
try:
# Parse input data
logger.info("Parsing input data")
if isinstance(input_data, str):
try:
data = json.loads(input_data)
if "input_data" in data:
data = data["input_data"]
except json.JSONDecodeError as e:
raise InputValidationError(f"Failed to parse JSON input: {str(e)}")
else:
data = input_data
# Validate input
self.validate_input(data)
# Extract summaries and key points
summary_text = ""
if "key_points" in data:
summary_text = "\n\n".join(data["key_points"])
elif "summary" in data:
summary_text = data["summary"]
logger.info(f"Extracted summary text: {summary_text[:100]}...")
# Verify API key
if not self.api_key:
self.api_key = os.getenv("OPENAI_API_KEY")
if not self.api_key:
raise APIError("OpenAI API key not found in environment variables")
client = OpenAI(api_key=self.api_key)
prompt = self._create_prompt(summary_text)
try:
response = client.chat.completions.create(
model="gpt-3.5-turbo",
messages=[
{
"role": "system",
"content": "You are an expert analyst specializing in identifying trends, implications, and business opportunities from news data."
},
{"role": "user", "content": prompt}
],
max_tokens=1000,
temperature=0.7
)
except Exception as e:
raise APIError(f"OpenAI API call failed: {str(e)}")
analysis = response.choices[0].message.content.strip()
logger.info("Successfully received analysis from OpenAI")
return self._parse_analysis(analysis)
except InputValidationError as e:
logger.error(f"Input validation error: {str(e)}")
return self._error_response(f"Input validation error: {str(e)}")
except APIError as e:
logger.error(f"API error: {str(e)}")
return self._error_response(f"API error: {str(e)}")
except Exception as e:
logger.error(f"Unexpected error: {str(e)}")
return self._error_response(f"Unexpected error: {str(e)}")
def _create_prompt(self, summary_text: str) -> str:
"""Creates the analysis prompt"""
return (
"Based on these news summaries:\n\n"
f"{summary_text}\n\n"
"Please analyze and identify:\n"
"1. Key emerging trends\n"
"2. Potential future implications and outcomes\n"
"3. Business opportunities arising from these developments\n\n"
"Format your response exactly as follows (provide at least 3 of each):\n"
"Trend: [specific trend description]\n"
"Trend: [specific trend description]\n"
"Trend: [specific trend description]\n"
"Outcome: [specific outcome description]\n"
"Outcome: [specific outcome description]\n"
"Outcome: [specific outcome description]\n"
"Business Idea: [specific idea description]\n"
"Business Idea: [specific idea description]\n"
"Business Idea: [specific idea description]"
)
def _parse_analysis(self, analysis: str) -> Dict[str, list]:
"""Parses the analysis response"""
trends = [line.strip() for line in analysis.split('\n') if line.strip().startswith('Trend:')]
outcomes = [line.strip() for line in analysis.split('\n') if line.strip().startswith('Outcome:')]
business_ideas = [line.strip() for line in analysis.split('\n') if line.strip().startswith('Business Idea:')]
return {
"trends": trends or ["No clear trends identified"],
"outcomes": outcomes or ["No specific outcomes predicted"],
"business_ideas": business_ideas or ["No business ideas generated"]
}
def _error_response(self, error_message: str) -> Dict[str, list]:
"""Creates a standardized error response"""
return {
"trends": [f"Error analyzing trends: {error_message}"],
"outcomes": ["Analysis failed"],
"business_ideas": ["Analysis failed"]
}
async def _arun(self, input_data: Union[str, Dict]) -> Dict[str, list]:
"""Async version of the run method"""
return self._run(input_data)

View file

@ -0,0 +1,31 @@
news_collector:
role: >
{topic} Expert News Collector
goal: >
Gather the latest news and information on {topic}
backstory: >
You're a seasoned and expert news collector who gathers relevant news and information on {topic}. Known for your ability to find the most relevant information and present it in a clear and concise manner.
fact_checker:
role: >
{topic} Fact Checker
goal: >
Verify the accuracy of news and information related to {topic}
backstory: >
You're a diligent fact checker with a reputation for accuracy. Your expertise lies in verifying facts and ensuring that all information is credible and reliable.
summary_writer:
role: >
{topic} Summary Writer
goal: >
Create concise summaries of news articles and reports on {topic}
backstory: >
You're a skilled writer known for your ability to distill complex information into clear and concise summaries. Your work helps others quickly understand the key points of any news story.
trend_analyzer:
role: >
{topic} Trend Analyzer
goal: >
Analyze trends and patterns in news data related to {topic}
backstory: >
You're an insightful trend analyzer with a knack for identifying patterns and trends in large datasets. Your analyse help predict future developments and inform strategic decisions and potential Business Ideas.

View file

@ -0,0 +1,57 @@
research_task:
description: >
Collect the latest news and information on {topic}
Make sure you find any interesting and relevant information given
the current year is 2024.
expected_output: >
A list of dictionaries, each containing:
- "title": The title of the news article.
- "url": The URL where the article can be accessed.
- "source": The name of the publication or source.
agent: news_collector
fact_check_task:
description: >
Verify the accuracy of news articles collected on {topic}.
Use the Fact Checker tool for each article provided by the news collector.
Only return the titles of articles that are determined to be factual.
expected_output: >
A list of strings, where each string is the title of an article that has been verified as factual.
For example:
[
"AI Breakthrough in Medical Diagnosis",
"New Study Shows Promise in Quantum Computing",
"Tech Giants Collaborate on Ethical AI Guidelines"
]
agent: fact_checker
summary_task:
description: >
Create concise summaries of news articles and reports on {topic}
expected_output: >
A dictionary containing:
- "title": The title of the summary.
- "summary": A brief overview of the main findings or conclusions.
- "key_points": A list of key points or highlights from the summary.
- "sources": A list of URLs where the information was sourced.
agent: summary_writer
trend_task:
description: >
Analyze trends and patterns in news data related to {topic}
expected_output: >
A dictionary containing:
- "main_trend": A brief description of the primary trend identified.
- "sub_trends": A list of sub-trends related to the main trend.
- "sentiment_analysis": A dictionary with:
- "overall_sentiment": A numerical representation of the overall sentiment.
- "sentiment_breakdown": A dictionary with percentages of positive, neutral, and negative sentiments.
- "key_entities": A list of dictionaries, each with:
- "name": The name of the entity.
- "type": The type of entity (e.g., Technology, Person).
- "frequency": The frequency of the entity's mention.
- "timeline": A list of dictionaries, each with:
- "date": The date of the event in ISO 8601 format.
- "event": A brief description of the event.
agent: trend_analyzer

View file

@ -0,0 +1,116 @@
from crewai import Agent, Crew, Process, Task
from langchain_openai import ChatOpenAI
from agents.news_collector import NewsAPITool
from agents.fact_checker import FactCheckerTool
from agents.summary_writer import SummaryWriterTool
from agents.trend_analyzer import TrendAnalyzerTool
import os
class NewsSummarizerAnalyzerCrew:
def __init__(self):
openai_api_key = os.getenv("OPENAI_API_KEY")
if not openai_api_key:
raise ValueError("OPENAI_API_KEY environment variable is not set")
self.llm = ChatOpenAI(temperature=0.7, model_name="gpt-3.5-turbo")
self.crew = self.create_crew()
def create_crew(self) -> Crew:
return Crew(
agents=[
self.news_collector(),
self.fact_checker(),
self.summary_writer(),
self.trend_analyzer()
],
tasks=[
self.news_collector_task(),
self.fact_checker_task(),
self.summary_writer_task(),
self.trend_analyzer_task()
],
process=Process.sequential,
verbose=True
)
def news_collector(self) -> Agent:
return Agent(
name="News Collector",
role="Collects news articles",
goal="Gather relevant news articles",
backstory="An expert in finding and collecting news from various sources",
tools=[NewsAPITool()],
verbose=True,
llm=self.llm
)
def fact_checker(self) -> Agent:
return Agent(
name="Fact Checker",
role="Verifies information in news articles",
goal="Ensure the accuracy of collected news",
backstory="A meticulous fact-checker with years of experience",
tools=[FactCheckerTool()],
verbose=True,
llm=self.llm,
custom_prompt="When using the Fact Checker tool, always format the input as a list of dictionaries. Each dictionary should have 'title', 'url', and 'snippet' keys."
)
def summary_writer(self) -> Agent:
return Agent(
name="Summary Writer",
role="Summarizes news articles",
goal="Create concise and informative summaries",
backstory="A skilled writer with a talent for distilling complex information",
tools=[SummaryWriterTool()],
verbose=True,
llm=self.llm
)
def trend_analyzer(self) -> Agent:
return Agent(
name="Trend Analyzer",
role="Analyzes trends in news",
goal="Identify and report on emerging trends",
backstory="An expert in data analysis and trend forecasting",
tools=[TrendAnalyzerTool()],
verbose=True,
llm=self.llm
)
def news_collector_task(self) -> Task:
return Task(
description="Gather the latest relevant news articles on the specified topic: {topic}",
agent=self.news_collector(),
expected_output="A list of dictionaries, each containing 'title', 'url', and 'snippet' keys for each relevant news article."
)
def fact_checker_task(self) -> Task:
return Task(
description="Verify the accuracy of news articles collected on the topic. Use the Fact Checker tool with a list of article dictionaries as input. Each article dictionary should have 'title', 'url', and 'snippet' keys.",
agent=self.fact_checker(),
expected_output="A list of strings, where each string is the title of an article that has been verified as factual."
)
def summary_writer_task(self) -> Task:
return Task(
description="Write concise summaries of the verified news articles. Each article is represented by its title.",
agent=self.summary_writer(),
expected_output="""
A dictionary containing:
- "title": The title of the summary.
- "summary": A brief overview of the main findings or conclusions.
- "key_points": A list of key points or highlights from the summary.
- "sources": A list of article titles that were summarized.
"""
)
def trend_analyzer_task(self) -> Task:
return Task(
description="Analyze trends in the summarized news. The input will be a dictionary containing the summary of all topics.",
agent=self.trend_analyzer(),
expected_output="An insightful analysis report identifying emerging trends, patterns, and potential future developments based on the summarized news data, including strategic implications and potential business ideas."
)
def kickoff(self, topic):
return self.crew.kickoff(inputs={"topic": topic})

View file

@ -0,0 +1,72 @@
import os
from dotenv import load_dotenv
from fastapi import FastAPI, HTTPException
from fastapi.middleware.cors import CORSMiddleware
from pydantic import BaseModel
from crew import NewsSummarizerAnalyzerCrew
import logging
# Set up logging
logging.basicConfig(level=logging.INFO)
logger = logging.getLogger(__name__)
load_dotenv()
app = FastAPI()
# Add CORS middleware with both localhost and deployed URLs
app.add_middleware(
CORSMiddleware,
allow_origins=[
"https://ai-business-insider.streamlit.app",
"http://localhost:8501",
],
allow_credentials=True,
allow_methods=["*"],
allow_headers=["*"],
)
# Health check endpoint
@app.get("/health")
async def health_check():
logger.info("Health check requested")
return {
"status": "healthy",
"environment": os.getenv("ENVIRONMENT", "production")
}
crew = NewsSummarizerAnalyzerCrew()
class TopicRequest(BaseModel):
topic: str
@app.post("/analyze")
async def analyze_topic(request: TopicRequest) -> dict:
"""
Analyzes a given topic using the NewsSummarizerAnalyzerCrew.
"""
try:
logger.info(f"Analyzing topic: {request.topic}")
crew_output = crew.kickoff(request.topic)
result = {
"summary": crew_output.raw,
"tasks": [
{
"description": task.description,
"output": task.raw
}
for task in crew_output.tasks_output
]
}
logger.info("Analysis completed successfully")
return result
except Exception as e:
logger.error(f"Error during analysis: {str(e)}")
raise HTTPException(status_code=500, detail=str(e))
if __name__ == "__main__":
import uvicorn
port = int(os.getenv("PORT", 8000))
uvicorn.run(app, host="0.0.0.0", port=port)

View file

@ -0,0 +1,61 @@
import streamlit as st
import requests
import os
from dotenv import load_dotenv
# Load environment variables
load_dotenv()
BACKEND_URL = os.getenv("BACKEND_URL", "http://localhost:8000")
st.title("The AI Business Insider")
st.sidebar.markdown("### System Status")
try:
health_check = requests.get(f"{BACKEND_URL}/health", timeout=5)
if health_check.ok:
st.sidebar.success("✅ Backend Connected")
st.sidebar.info(f"Environment: {health_check.json().get('environment', 'unknown')}")
else:
st.sidebar.error("❌ Backend Error")
except Exception as e:
st.sidebar.error("❌ Backend Not Connected")
st.sidebar.info(f"Backend URL: {BACKEND_URL}")
# Get user input
topic = st.text_input("Enter a topic to analyze:", key="topic_input")
if st.button("Analyze"):
if topic:
with st.spinner("Analyzing..."):
try:
response = requests.post(
f"{BACKEND_URL}/analyze",
json={"topic": topic},
timeout=300 # 5 minute timeout
)
response.raise_for_status()
result = response.json()
st.success("Analysis Complete!")
# Display summary
st.header("Summary")
st.write(result["summary"])
# Display individual task results
st.header("Detailed Analysis")
for task in result["tasks"]:
with st.expander(f"Task: {task['description']}", expanded=False):
st.write(task["output"])
except requests.exceptions.ConnectionError:
st.error(f"Cannot connect to backend at {BACKEND_URL}. Please ensure the backend server is running.")
except requests.exceptions.Timeout:
st.error("Request timed out. Please try again.")
except requests.exceptions.RequestException as e:
st.error(f"Error communicating with backend: {str(e)}")
except Exception as e:
st.error(f"An error occurred: {str(e)}")
else:
st.warning("Please enter a topic to analyze")

View file

@ -0,0 +1,29 @@
from agents.trend_analyzer import TrendAnalyzerTool
from dotenv import load_dotenv
import json
import os
# Load environment variables
load_dotenv()
def test_specific_input():
# The exact input you provided
test_input = """{\"input_data\": {\"title\": \"Summaries of Verified News Articles\", \"summary\": \"1. Trump's crypto website experienced a crash following the sale of its token. 2. Scotland's Treasury demands compensation for a tax increase. 3. The launch of Trumpcoin faced a lackluster response. 4. UK borrowing in September reached the third-highest level on record. 5. The banking industry creator emphasizes that bankers are neither seen as villains nor rock stars.\", \"key_points\": [\"Trump's crypto website crash post-token sale\", \"Scotland's Treasury seeks compensation for tax hike\", \"Trumpcoin launch met with low interest\", \"UK records third-highest borrowing in September\", \"Bankers perceived as neither villains nor rock stars\"], \"sources\": [\"Trump\\u2019s crypto website crashed after its token went on sale\", \"Treasury must compensate Scotland for tax hike - Robison\", \"Trumpcoin Launches With a Whimper\", \"UK borrowing for September third highest on record\", \"Bankers 'neither villains nor rock stars', says Industry creator\"]}}"""
print("\n=== Testing Specific Input ===")
print("\nInput data (formatted):")
parsed = json.loads(test_input)
print(json.dumps(parsed, indent=2))
analyzer = TrendAnalyzerTool()
print("\nProcessing...")
result = analyzer._run(test_input)
print("\nResult:")
print(json.dumps(result, indent=2))
print("\n=== Test Complete ===")
if __name__ == "__main__":
test_specific_input()

View file

@ -0,0 +1,39 @@
from crewai_tools import BaseTool
import requests
from typing import List, Dict, Literal
from pydantic import Field
class NewsAPITool(BaseTool):
name: str = "News API"
description: str = (
"Fetches the latest news articles based on a given query using the News API."
)
api_key: str = Field(..., description="API key for the News API")
base_url: Literal["https://newsapi.org/v2/everything"] = "https://newsapi.org/v2/everything"
def _run(self, query: str) -> List[Dict[str, str]]:
"""Fetches news articles based on a query"""
params = {
'q': query,
'apiKey': self.api_key
}
try:
response = requests.get(self.base_url, params=params)
response.raise_for_status()
articles = response.json().get('articles', [])
return [
{
"title": article["title"],
"url": article["url"],
"source": article["source"]["name"],
"published_date": article["publishedAt"],
"snippet": article["description"]
}
for article in articles[:10]
]
except requests.RequestException as e:
raise Exception(f"Error fetching news articles: {str(e)}")
async def _arun(self, query: str) -> List[Dict[str, str]]:
"""Async implementation of the tool"""
return self._run(query)

View file

@ -0,0 +1,37 @@
import os
import sys
from dotenv import load_dotenv
# Load environment variables from .env file
load_dotenv()
# Add the parent directory of 'news_summarizer_analyzer' to sys.path
parent_dir = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))
sys.path.insert(0, parent_dir)
try:
from src.news_summarizer_analyzer.tools.custom_tool import NewsAPITool
print("Import successful")
except ImportError as e:
print(f"Import failed: {e}")
def test_news_api_tool():
api_key: str = os.getenv("NEWS_API_KEY")
if not api_key:
raise ValueError("NEWS_API_KEY environment variable is not set")
tool: NewsAPITool = NewsAPITool(api_key=api_key)
# Define a query to test
query: str = 'Financial Markets'
try:
# Fetch articles
articles: list = tool._run(query)
for article in articles:
print(f"Title: {article['title']}, Source: {article['source']}")
except Exception as e:
print(f"An error occurred: {e}")
if __name__ == "__main__":
test_news_api_tool()

View file

@ -0,0 +1,39 @@
import os
import sys
from dotenv import load_dotenv
# Load environment variables from .env file
load_dotenv()
# Add the parent directory of 'news_summarizer_analyzer' to sys.path
parent_dir = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))
sys.path.insert(0, parent_dir)
try:
from src.news_summarizer_analyzer.agents.fact_checker import FactCheckerTool
print("Import successful")
except ImportError as e:
print(f"Import failed: {e}")
def test_fact_checker():
api_key: str = os.getenv("GOOGLE_FACT_CHECK_API_KEY")
if not api_key:
raise ValueError("GOOGLE_FACT_CHECK_API_KEY environment variable is not set")
fact_checker = FactCheckerTool(api_key=api_key)
# Example article to test
article = {
"title": "AI Breakthrough in Climate Modeling",
"snippet": "Researchers use advanced AI to improve climate change predictions..."
}
try:
# Run the fact checker
result = fact_checker._run(article)
print("Fact Check Result:")
print(result)
except Exception as e:
print(f"An error occurred: {e}")
if __name__ == "__main__":
test_fact_checker()

View file

@ -0,0 +1,39 @@
import os
import sys
from dotenv import load_dotenv
# Load environment variables from .env file
load_dotenv()
# Add the parent directory of 'news_summarizer_analyzer' to sys.path
parent_dir = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))
sys.path.insert(0, parent_dir)
try:
from src.news_summarizer_analyzer.agents.summary_writer import SummaryWriterTool
print("Import successful")
except ImportError as e:
print(f"Import failed: {e}")
def test_summary_writer():
api_key: str = os.getenv("OPENAI_API_KEY")
if not api_key:
raise ValueError("OPENAI_API_KEY environment variable is not set")
summary_writer = SummaryWriterTool(api_key=api_key)
# Example article to test
article = {
"title": "AI Breakthrough in Climate Modeling",
"snippet": "Researchers use advanced AI to improve climate change predictions..."
}
try:
# Run the fact checker
result = summary_writer._run(article)
print("Summary Writer Result:")
print(result)
except Exception as e:
print(f"An error occurred: {e}")
if __name__ == "__main__":
test_summary_writer()

View file

@ -0,0 +1,303 @@
aiohappyeyeballs==2.4.3
aiohttp==3.10.10
aiosignal==1.3.1
alembic==1.13.3
altair==5.4.1
aniso8601==9.0.1
annotated-types==0.7.0
anyio==4.6.0
appdirs==1.4.4
asgiref==3.8.1
asttokens==2.4.1
async-timeout==4.0.3
attrs==23.2.0
auth0-python==4.7.2
backoff==2.2.1
bcrypt==4.2.0
beautifulsoup4==4.12.3
blinker==1.8.2
boto3==1.35.39
botocore==1.35.39
build==1.2.2.post1
CacheControl==0.14.0
cachetools==5.5.0
certifi==2024.8.30
cffi==1.17.1
charset-normalizer==3.4.0
chroma-hnswlib==0.7.3
chromadb==0.4.24
cleo==2.1.0
click==8.1.7
cloudpickle==2.2.1
cohere==5.11.0
coloredlogs==15.0.1
contourpy==1.3.0
crashtest==0.4.1
crewai==0.74.2
crewai-tools==0.13.2
cryptography==43.0.1
cycler==0.12.1
databricks-sdk==0.34.0
dataclasses-json==0.6.7
decorator==5.1.1
Deprecated==1.2.14
deprecation==2.1.0
dill==0.3.9
distlib==0.3.9
distro==1.9.0
docker==7.1.0
docstring_parser==0.16
docx2txt==0.8
dulwich==0.21.7
durationpy==0.9
embedchain==0.1.123
exceptiongroup==1.2.2
executing==2.1.0
fastapi==0.115.2
fastavro==1.9.7
fastjsonschema==2.20.0
filelock==3.16.1
Flask==3.0.3
flatbuffers==24.3.25
fonttools==4.54.1
frozenlist==1.4.1
fsspec==2024.9.0
gitdb==4.0.11
GitPython==3.1.43
google-ai-generativelanguage==0.6.10
google-api-core==2.21.0
google-api-python-client==2.149.0
google-auth==2.35.0
google-auth-httplib2==0.2.0
google-cloud-aiplatform==1.70.0
google-cloud-bigquery==3.26.0
google-cloud-core==2.4.1
google-cloud-resource-manager==1.12.5
google-cloud-storage==2.18.2
google-crc32c==1.6.0
google-generativeai==0.8.3
google-pasta==0.2.0
google-resumable-media==2.7.2
googleapis-common-protos==1.65.0
gptcache==0.1.44
graphene==3.3
graphql-core==3.2.4
graphql-relay==3.2.0
grpc-google-iam-v1==0.13.1
grpcio==1.66.2
grpcio-status==1.62.3
grpcio-tools==1.62.3
gunicorn==23.0.0
h11==0.14.0
h2==4.1.0
hpack==4.0.0
httpcore==1.0.6
httplib2==0.22.0
httptools==0.6.1
httpx==0.27.2
httpx-sse==0.4.0
huggingface-hub==0.25.2
humanfriendly==10.0
hyperframe==6.0.1
idna==3.10
importlib-metadata==6.11.0
importlib_resources==6.4.5
iniconfig==2.0.0
installer==0.7.0
instructor==1.3.3
ipython==8.28.0
itsdangerous==2.2.0
jaraco.classes==3.4.0
jedi==0.19.1
Jinja2==3.1.4
jiter==0.4.2
jmespath==1.0.1
joblib==1.4.2
json_repair==0.25.3
jsonpatch==1.33
jsonpickle==3.3.0
jsonpointer==3.0.0
jsonref==1.1.0
jsonschema==4.23.0
jsonschema-specifications==2024.10.1
keyring==24.3.1
kiwisolver==1.4.7
kubernetes==31.0.0
lancedb==0.5.7
langchain==0.3.4
langchain-cohere==0.3.1
langchain-community==0.3.3
langchain-core==0.3.12
langchain-experimental==0.3.2
langchain-google-vertexai==2.0.5
langchain-openai==0.2.3
langchain-text-splitters==0.3.0
langsmith==0.1.134
litellm==1.49.2
Mako==1.3.5
Markdown==3.7
markdown-it-py==3.0.0
MarkupSafe==3.0.1
marshmallow==3.22.0
matplotlib==3.9.2
matplotlib-inline==0.1.7
mdurl==0.1.2
mem0ai==0.1.21
mlflow==2.17.0
mlflow-skinny==2.17.0
mmh3==5.0.1
mock==4.0.3
monotonic==1.6
more-itertools==10.5.0
mpmath==1.3.0
msgpack==1.1.0
multidict==6.1.0
multiprocess==0.70.17
mypy-extensions==1.0.0
narwhals==1.13.2
neo4j==5.25.0
networkx==3.4.1
newsapi-python==0.2.7
nodeenv==1.9.1
numpy==1.26.4
oauthlib==3.2.2
onnxruntime==1.19.2
openai==1.52.0
opentelemetry-api==1.27.0
opentelemetry-exporter-otlp-proto-common==1.27.0
opentelemetry-exporter-otlp-proto-grpc==1.27.0
opentelemetry-exporter-otlp-proto-http==1.27.0
opentelemetry-instrumentation==0.48b0
opentelemetry-instrumentation-asgi==0.48b0
opentelemetry-instrumentation-fastapi==0.48b0
opentelemetry-proto==1.27.0
opentelemetry-sdk==1.27.0
opentelemetry-semantic-conventions==0.48b0
opentelemetry-util-http==0.48b0
orjson==3.10.7
outcome==1.3.0.post0
overrides==7.7.0
packaging==24.1
pandas==2.2.3
parameterized==0.9.0
parso==0.8.4
pathos==0.3.3
pexpect==4.9.0
pillow==10.4.0
pkginfo==1.11.2
platformdirs==4.3.6
pluggy==1.5.0
poetry==1.8.3
poetry-core==1.9.0
poetry-plugin-export==1.8.0
portalocker==2.10.1
posthog==3.7.0
pox==0.3.5
ppft==1.7.6.9
prompt_toolkit==3.0.48
propcache==0.2.0
proto-plus==1.24.0
protobuf==4.25.5
psutil==6.0.0
ptyprocess==0.7.0
pulsar-client==3.5.0
pure_eval==0.2.3
py==1.11.0
pyarrow==17.0.0
pyasn1==0.6.1
pyasn1_modules==0.4.1
pycparser==2.22
pydantic==2.9.2
pydantic-settings==2.6.0
pydantic_core==2.23.4
pydeck==0.9.1
Pygments==2.18.0
PyJWT==2.9.0
pylance==0.9.18
pyparsing==3.1.4
pypdf==5.0.1
PyPika==0.48.9
pyproject_hooks==1.2.0
pyright==1.1.384
pysbd==0.3.4
PySocks==1.7.1
pytest==8.3.3
python-dateutil==2.9.0.post0
python-dotenv==1.0.1
pytube==15.0.0
pytz==2024.2
pyvis==0.3.2
PyYAML==6.0.2
qdrant-client==1.12.0
rank-bm25==0.2.2
RapidFuzz==3.10.0
ratelimiter==1.2.0.post0
referencing==0.35.1
regex==2024.9.11
requests==2.32.3
requests-oauthlib==2.0.0
requests-toolbelt==1.0.0
retry==0.9.2
rich==13.9.2
rpds-py==0.20.0
rsa==4.9
s3transfer==0.10.3
sagemaker==2.232.2
sagemaker-core==1.0.10
sagemaker-mlflow==0.1.0
schema==0.7.7
scikit-learn==1.5.2
scipy==1.14.1
selenium==4.25.0
semver==3.0.2
shapely==2.0.6
shellingham==1.5.4
six==1.16.0
smdebug-rulesconfig==1.0.1
smmap==5.0.1
sniffio==1.3.1
sortedcontainers==2.4.0
soupsieve==2.6
SQLAlchemy==2.0.35
sqlparse==0.5.1
stack-data==0.6.3
starlette==0.39.2
streamlit==1.39.0
sympy==1.13.3
tabulate==0.9.0
tblib==3.0.0
tenacity==8.5.0
threadpoolctl==3.5.0
tiktoken==0.7.0
tokenizers==0.20.1
toml==0.10.2
tomli==2.0.2
tomli_w==1.1.0
tomlkit==0.13.2
tornado==6.4.1
tqdm==4.66.5
traitlets==5.14.3
trio==0.26.2
trio-websocket==0.11.1
trove-classifiers==2024.10.12
typer==0.12.5
types-requests==2.32.0.20240914
typing-inspect==0.9.0
typing_extensions==4.12.2
tzdata==2024.2
uritemplate==4.1.1
urllib3==2.2.3
uv==0.4.24
uvicorn==0.31.1
uvloop==0.20.0
virtualenv==20.26.6
watchfiles==0.24.0
wcwidth==0.2.13
websocket-client==1.8.0
websockets==13.1
Werkzeug==3.0.4
wrapt==1.16.0
wsproto==1.2.0
xattr==1.1.0
yarl==1.15.0
zipp==3.20.2