LangChain
Use o Firecrawl com o LangChain para web scraping e fluxos de IA
Integre o Firecrawl ao LangChain para criar aplicativos de IA alimentados por dados da web.
npm install @langchain/openai firecrawl Crie o arquivo .env:
FIRECRAWL_API_KEY=your_firecrawl_key
OPENAI_API_KEY=your_openai_keyObservação: Se estiver usando Node < 20, instale
dotenve adicioneimport 'dotenv/config'ao seu código.
Este exemplo demonstra um fluxo de trabalho simples: fazer scraping de um site e processar o conteúdo com o LangChain.
import { Firecrawl } from 'firecrawl';
import { ChatOpenAI } from '@langchain/openai';
import { HumanMessage } from '@langchain/core/messages';
const firecrawl = new Firecrawl({ apiKey: process.env.FIRECRAWL_API_KEY });
const chat = new ChatOpenAI({
model: 'gpt-5-nano',
apiKey: process.env.OPENAI_API_KEY
});
const scrapeResult = await firecrawl.scrape('https://firecrawl.dev', {
formats: ['markdown']
});
console.log('Scraped content length:', scrapeResult.markdown?.length);
const response = await chat.invoke([
new HumanMessage(`Summarize: ${scrapeResult.markdown}`)
]);
console.log('Summary:', response.content);Este exemplo mostra como criar uma chain no LangChain para processar e analisar o conteúdo extraído.
import { Firecrawl } from 'firecrawl';
import { ChatOpenAI } from '@langchain/openai';
import { ChatPromptTemplate } from '@langchain/core/prompts';
import { StringOutputParser } from '@langchain/core/output_parsers';
const firecrawl = new Firecrawl({ apiKey: process.env.FIRECRAWL_API_KEY });
const model = new ChatOpenAI({
model: 'gpt-5-nano',
apiKey: process.env.OPENAI_API_KEY
});
const scrapeResult = await firecrawl.scrape('https://stripe.com', {
formats: ['markdown']
});
console.log('Scraped content length:', scrapeResult.markdown?.length);
// Criar chain de processamento
const prompt = ChatPromptTemplate.fromMessages([
['system', 'You are an expert at analyzing company websites.'],
['user', 'Extract the company name and main products from: {content}']
]);
const chain = prompt.pipe(model).pipe(new StringOutputParser());
// Executar a chain
const result = await chain.invoke({
content: scrapeResult.markdown
});
console.log('Chain result:', result);Este exemplo demonstra como usar o recurso de chamada de ferramentas do LangChain para permitir que o modelo decida quando fazer scraping de sites.
import { Firecrawl } from 'firecrawl';
import { ChatOpenAI } from '@langchain/openai';
import { DynamicStructuredTool } from '@langchain/core/tools';
import { z } from 'zod';
const firecrawl = new Firecrawl({ apiKey: process.env.FIRECRAWL_API_KEY });
// Cria a ferramenta de scraping
const scrapeWebsiteTool = new DynamicStructuredTool({
name: 'scrape_website',
description: 'Scrape content from any website URL',
schema: z.object({
url: z.string().url().describe('The URL to scrape')
}),
func: async ({ url }) => {
console.log('Scraping:', url);
const result = await firecrawl.scrape(url, {
formats: ['markdown']
});
console.log('Scraped content preview:', result.markdown?.substring(0, 200) + '...');
return result.markdown || 'No content scraped';
}
});
const model = new ChatOpenAI({
model: 'gpt-5-nano',
apiKey: process.env.OPENAI_API_KEY
}).bindTools([scrapeWebsiteTool]);
const response = await model.invoke('What is Firecrawl? Visit firecrawl.dev and tell me about it.');
console.log('Response:', response.content);
console.log('Tool calls:', response.tool_calls);Este exemplo mostra como extrair dados estruturados usando a funcionalidade de saída estruturada do LangChain.
import { Firecrawl } from 'firecrawl';
import { ChatOpenAI } from '@langchain/openai';
import { z } from 'zod';
const firecrawl = new Firecrawl({ apiKey: process.env.FIRECRAWL_API_KEY });
const scrapeResult = await firecrawl.scrape('https://stripe.com', {
formats: ['markdown']
});
console.log('Tamanho do conteúdo extraído:', scrapeResult.markdown?.length);
const CompanyInfoSchema = z.object({
name: z.string(),
industry: z.string(),
description: z.string(),
products: z.array(z.string())
});
const model = new ChatOpenAI({
model: 'gpt-5-nano',
apiKey: process.env.OPENAI_API_KEY
}).withStructuredOutput(CompanyInfoSchema);
const companyInfo = await model.invoke([
{
role: 'system',
content: 'Extraia informações da empresa do conteúdo do site.'
},
{
role: 'user',
content: `Extraia os dados: ${scrapeResult.markdown}`
}
]);
console.log('Informações da empresa extraídas:', companyInfo);Para mais exemplos, consulte a documentação do LangChain.