你当前正在访问 Microsoft Azure Global Edition 技术文档网站。 如果需要访问由世纪互联运营的 Microsoft Azure 中国技术文档网站,请访问 https://docs.azure.cn。

教程:构建文档总结器

生成一个应用程序,用于读取文本文件并生成简洁的摘要 - 完全在设备上。 当你需要快速了解文档的内容而不完全阅读文档时,以及文档包含不应离开计算机的敏感信息时,这非常有用。

本教程中,您将学习如何:

  • 设置项目并安装 Foundry 本地 SDK
  • 从文件系统读取文本文档
  • 加载模型并生成摘要
  • 使用系统提示控制摘要输出
  • 批量处理多个文档
  • 清理资源

先决条件

  • 至少具有 8 GB RAM 的 Windows、macOS 或 Linux 计算机。

示例存储库

可以在 Foundry 示例GitHub存储库中找到本文的完整示例代码。 若要克隆存储库并导航到示例,请使用:

git clone https://github.com/microsoft-foundry/foundry-samples.git
cd foundry-samples/samples/csharp/foundry-local/tutorial-document-summarizer

安装软件包

dotnet add package Microsoft.AI.Foundry.Local
dotnet add package OpenAI

GitHub存储库中的 C# 示例是预配置项目。 如果要从头开始生成,则应阅读 Foundry Local SDK 参考 ,详细了解如何使用 Foundry Local 设置 C# 项目。

读取文本文档

在汇总任何内容之前,需要一个示例文档来处理。 在项目目录中创建一个调用 document.txt 的文件,并添加以下内容:

Automated testing is a practice in software development where tests are written and executed
by specialized tools rather than performed manually. There are several categories of automated
tests, including unit tests, integration tests, and end-to-end tests. Unit tests verify that
individual functions or methods behave correctly in isolation. Integration tests check that
multiple components work together as expected. End-to-end tests simulate real user workflows
across the entire application.

Adopting automated testing brings measurable benefits to a development team. It catches
regressions early, before they reach production. It reduces the time spent on repetitive
manual verification after each code change. It serves as living documentation of expected
behavior, which helps new team members understand the codebase. Continuous integration
pipelines rely on automated tests to gate deployments and maintain release quality.

Effective test suites follow a few guiding principles. Tests should be deterministic, meaning
they produce the same result every time they run. Tests should be independent, so that one
failing test does not cascade into false failures elsewhere. Tests should run fast, because
slow tests discourage developers from running them frequently. Finally, tests should be
maintained alongside production code so they stay accurate as the application evolves.

现在打开 Program.cs 并添加以下代码以读取文档:

var target = args.Length > 0 ? args[0] : "document.txt";

代码接受一个可选的文件路径作为命令行参数,如果未提供路径,则回退到document.txt。

生成摘要

初始化 Foundry Local SDK,加载模型,并发送文档内容,同时附带系统提示来让模型进行总结。

将 Program.cs 的内容替换为以下代码:

var systemPrompt =
    "Summarize the following document into concise bullet points. " +
    "Focus on the key points and main ideas.";

var target = args.Length > 0 ? args[0] : "document.txt";

if (Directory.Exists(target))
{
    await SummarizeDirectoryAsync(model, target, systemPrompt, ct);
}
else
{
    Console.WriteLine($"--- {Path.GetFileName(target)} ---");
    await SummarizeFileAsync(model, target, systemPrompt, ct);
}

该方法 GetModelAsync 接受模型别名,该别名是映射到目录中特定模型的短友好名称。 方法DownloadAsync 会将模型的权重提取到本地缓存中(如果已存在缓存会跳过下载),方法LoadAsync 让模型可以进行推理准备。 系统提示要求模型生成侧重于关键想法的要点摘要。

控制摘要输出

不同的情况要求使用不同的摘要样式。 可以更改系统提示以控制模型如何构建其输出。 下面是三个有用的变体:

项目符号点(上一步的默认值):

var systemPrompt =
    "Summarize the following document into concise bullet points. " +
    "Focus on the key points and main ideas.";

一段摘要:

var systemPrompt =
    "Summarize the following document in a single, concise paragraph. " +
    "Capture the main argument and supporting points.";

要点:

var systemPrompt =
    "Extract the three most important takeaways from the following document. " +
    "Number each takeaway and keep each to one or two sentences.";

若要尝试不同的样式,请将系统消息中的Content值替换为其中一个提示。 该模型遵循系统提示中的说明来塑造摘要的格式和深度。

处理多个文档

将应用程序扩展以总结目录中的每个 .txt 文件。 当你有一个全部需要摘要的文档文件夹时,这非常有用。

以下方法循环访问给定目录中的所有 .txt 文件,并汇总每个文件:

async Task SummarizeDirectoryAsync(
    dynamic chatClient,
    string directory,
    string systemPrompt,
    CancellationToken ct)
{
    var txtFiles = Directory.GetFiles(directory, "*.txt")
        .OrderBy(f => f)
        .ToArray();

    if (txtFiles.Length == 0)
    {
        Console.WriteLine($"No .txt files found in {directory}");
        return;
    }

    foreach (var txtFile in txtFiles)
    {
        var fileContent = await File.ReadAllTextAsync(txtFile, ct);
        var msgs = new List<ChatMessage>
        {
            new ChatMessage { Role = "system", Content = systemPrompt },
            new ChatMessage { Role = "user", Content = fileContent }
        };

        Console.WriteLine($"--- {Path.GetFileName(txtFile)} ---");
        var resp = await chatClient.CompleteChatAsync(msgs, ct);
        Console.WriteLine(resp.Choices[0].Message.Content);
        Console.WriteLine();
    }
}

每个文件都会被读取,与同一系统提示进行配对,然后独立地发送到模型。 模型不会在文件之间携带上下文,因此每个摘要都是自包含的。

完整代码

将 Program.cs 的内容替换为完整的代码:

using Microsoft.AI.Foundry.Local;
using Microsoft.Extensions.Logging;

CancellationToken ct = CancellationToken.None;

var config = new Configuration
{
    AppName = "foundry_local_samples",
    LogLevel = Microsoft.AI.Foundry.Local.LogLevel.Information
};

using var loggerFactory = LoggerFactory.Create(builder =>
{
    builder.SetMinimumLevel(Microsoft.Extensions.Logging.LogLevel.Information);
});
var logger = loggerFactory.CreateLogger<Program>();

// Initialize the singleton instance
await FoundryLocalManager.CreateAsync(config, logger);
var mgr = FoundryLocalManager.Instance;

// Download and register all execution providers.
var currentEp = "";
await mgr.DownloadAndRegisterEpsAsync((epName, percent) =>
{
    if (epName != currentEp)
    {
        if (currentEp != "") Console.WriteLine();
        currentEp = epName;
    }
    Console.Write($"\r  {epName.PadRight(30)}  {percent,6:F1}%");
});
if (currentEp != "") Console.WriteLine();

// Select and load a model from the catalog
var catalog = await mgr.GetCatalogAsync();
var model = await catalog.GetModelAsync("qwen2.5-0.5b")
    ?? throw new Exception("Model not found");

await model.DownloadAsync(progress =>
{
    Console.Write($"\rDownloading model: {progress:F2}%");
    if (progress >= 100f) Console.WriteLine();
});

await model.LoadAsync();
Console.WriteLine("Model loaded and ready.\n");


var systemPrompt =
    "Summarize the following document into concise bullet points. " +
    "Focus on the key points and main ideas.";

var target = args.Length > 0 ? args[0] : "document.txt";

if (Directory.Exists(target))
{
    await SummarizeDirectoryAsync(model, target, systemPrompt, ct);
}
else
{
    Console.WriteLine($"--- {Path.GetFileName(target)} ---");
    await SummarizeFileAsync(model, target, systemPrompt, ct);
}

// Clean up
await model.UnloadAsync();
Console.WriteLine("\nModel unloaded. Done!");

async Task SummarizeFileAsync(
    IModel loadedModel,
    string filePath,
    string prompt,
    CancellationToken token)
{
    var fileContent = await File.ReadAllTextAsync(filePath, token);
    using var session = new ChatSession(loadedModel);
    using var request = new Request();
    request.AddItem(MessageItem.System(prompt));
    request.AddItem(MessageItem.User(fileContent));
    using var response = await session.ProcessRequestAsync(request, token);
    Console.WriteLine(string.Concat(
        response.OfType<MessageItem>()
            .SelectMany(message => message.Parts)
            .OfType<TextItem>()
            .Where(text => text.Type == TextItemType.Default)
            .Select(text => text.Text)));
}

async Task SummarizeDirectoryAsync(
    IModel loadedModel,
    string directory,
    string prompt,
    CancellationToken token)
{
    var txtFiles = Directory.GetFiles(directory, "*.txt")
        .OrderBy(f => f)
        .ToArray();

    if (txtFiles.Length == 0)
    {
        Console.WriteLine($"No .txt files found in {directory}");
        return;
    }

    foreach (var txtFile in txtFiles)
    {
        Console.WriteLine($"--- {Path.GetFileName(txtFile)} ---");
        await SummarizeFileAsync(loadedModel, txtFile, prompt, token);
        Console.WriteLine();
    }
}

汇总单个文件:

dotnet run -- document.txt

或汇总目录中的每个 .txt 文件:

dotnet run -- ./docs

会看到类似于以下内容的输出:

Downloading model: 100.00%
Model loaded and ready.

--- document.txt ---
- Automated testing uses specialized tools to execute tests instead of manual verification.
- Tests fall into three main categories: unit tests (individual functions), integration tests
  (component interactions), and end-to-end tests (full user workflows).
- Key benefits include catching regressions early, reducing manual effort, serving as living
  documentation, and gating deployments through continuous integration pipelines.
- Effective test suites should be deterministic, independent, fast, and maintained alongside
  production code.

Model unloaded. Done!

示例存储库

本文的完整示例代码在 foundry-samples GitHub 存储库中提供。 若要克隆存储库并导航到示例,请使用:

git clone https://github.com/microsoft-foundry/foundry-samples.git
cd foundry-samples/samples/javascript/foundry-local/tutorial-document-summarizer

安装软件包

npm install foundry-local-sdk openai

读取文本文档

在汇总任何内容之前,需要一个示例文档来处理。 在项目目录中创建一个调用 document.txt 的文件,并添加以下内容:

Automated testing is a practice in software development where tests are written and executed
by specialized tools rather than performed manually. There are several categories of automated
tests, including unit tests, integration tests, and end-to-end tests. Unit tests verify that
individual functions or methods behave correctly in isolation. Integration tests check that
multiple components work together as expected. End-to-end tests simulate real user workflows
across the entire application.

Adopting automated testing brings measurable benefits to a development team. It catches
regressions early, before they reach production. It reduces the time spent on repetitive
manual verification after each code change. It serves as living documentation of expected
behavior, which helps new team members understand the codebase. Continuous integration
pipelines rely on automated tests to gate deployments and maintain release quality.

Effective test suites follow a few guiding principles. Tests should be deterministic, meaning
they produce the same result every time they run. Tests should be independent, so that one
failing test does not cascade into false failures elsewhere. Tests should run fast, because
slow tests discourage developers from running them frequently. Finally, tests should be
maintained alongside production code so they stay accurate as the application evolves.

现在创建一个名为 index.js 的文件,并添加以下代码以读取文档:

const target = process.argv[2] || 'document.txt';

该脚本接受可选的文件路径作为命令行输入参数,如果未提供,则回退到document.txt。

生成摘要

初始化 Foundry Local SDK,加载模型,并发送文档内容,同时附带系统提示来让模型进行总结。

将 index.js 的内容替换为以下代码:

const systemPrompt =
    'Summarize the following document into concise bullet points. ' +
    'Focus on the key points and main ideas.';

const target = process.argv[2] || 'document.txt';

try {
    const stats = statSync(target);
    if (stats.isDirectory()) {
        await summarizeDirectory(model, target, systemPrompt);
    } else {
        console.log(`--- ${basename(target)} ---`);
        const session = new ChatSession(model);
        await summarizeFile(session, target, systemPrompt);
        session.dispose();
    }
} catch {
    console.log(`--- ${basename(target)} ---`);
    const session = new ChatSession(model);
    await summarizeFile(session, target, systemPrompt);
    session.dispose();
}

该方法 getModel 接受模型别名,该别名是映射到目录中特定模型的短友好名称。 方法download 会将模型的权重提取到本地缓存中(如果已存在缓存会跳过下载),方法load 让模型可以进行推理准备。 系统提示要求模型生成侧重于关键想法的要点摘要。

控制摘要输出

不同的情况要求使用不同的摘要样式。 可以更改系统提示以控制模型如何构建其输出。 下面是三个有用的变体:

项目符号点(上一步的默认值):

const systemPrompt =
    'Summarize the following document into concise bullet points. ' +
    'Focus on the key points and main ideas.';

一段摘要:

const systemPrompt =
    'Summarize the following document in a single, concise paragraph. ' +
    'Capture the main argument and supporting points.';

要点:

const systemPrompt =
    'Extract the three most important takeaways from the following document. ' +
    'Number each takeaway and keep each to one or two sentences.';

若要尝试不同的样式,请将系统消息中的content值替换为其中一个提示。 该模型遵循系统提示中的说明来塑造摘要的格式和深度。

处理多个文档

将应用程序扩展以总结目录中的每个 .txt 文件。 当你有一个全部需要摘要的文档文件夹时,这非常有用。

以下函数循环访问给定目录中的所有 .txt 文件,并汇总每个文件:

import { readdirSync } from 'fs';
import { join, basename } from 'path';

async function summarizeDirectory(chatClient, directory, systemPrompt) {
    const txtFiles = readdirSync(directory)
        .filter(f => f.endsWith('.txt'))
        .sort();

    if (txtFiles.length === 0) {
        console.log(`No .txt files found in ${directory}`);
        return;
    }

    for (const fileName of txtFiles) {
        const fileContent = readFileSync(join(directory, fileName), 'utf-8');
        const msgs = [
            { role: 'system', content: systemPrompt },
            { role: 'user', content: fileContent }
        ];

        console.log(`--- ${fileName} ---`);
        const resp = await chatClient.completeChat(msgs);
        console.log(resp.choices[0]?.message?.content);
        console.log();
    }
}

每个文件都会被读取,与同一系统提示进行配对,然后独立地发送到模型。 模型不会在文件之间携带上下文,因此每个摘要都是自包含的。

完整代码

创建一个名为 index.js 并添加以下完整代码的文件:

import {
    ChatSession,
    FoundryLocalManager,
    Item,
    Request
} from 'foundry-local-sdk';
import { readFileSync, readdirSync, statSync } from 'fs';
import { join, basename } from 'path';

async function summarizeFile(chatSession, filePath, systemPrompt) {
    const content = readFileSync(filePath, 'utf-8');
    const request = new Request()
        .addItem(Item.systemMessage(systemPrompt))
        .addItem(Item.userMessage(content));
    const response = await chatSession.processRequest(request);
    const text = response.output
        .map((item) => item.type === 'message'
            ? item.content ?? item.parts?.map((part) => part.text ?? '').join('') ?? ''
            : item.type === 'text' ? item.text : '')
        .join('');
    console.log(text);
}

async function summarizeDirectory(model, directory, systemPrompt) {
    const txtFiles = readdirSync(directory)
        .filter(f => f.endsWith('.txt'))
        .sort();

    if (txtFiles.length === 0) {
        console.log(`No .txt files found in ${directory}`);
        return;
    }

    for (const fileName of txtFiles) {
        console.log(`--- ${fileName} ---`);
        const session = new ChatSession(model);
        await summarizeFile(session, join(directory, fileName), systemPrompt);
        session.dispose();
        console.log();
    }
}

// Initialize the Foundry Local SDK
const manager = FoundryLocalManager.create({
    appName: 'foundry_local_samples',
    logLevel: 'info'
});

// Download and register all execution providers.
let currentEp = '';
await manager.downloadAndRegisterEps((epName, percent) => {
    if (epName !== currentEp) {
        if (currentEp !== '') process.stdout.write('\n');
        currentEp = epName;
    }
    process.stdout.write(`\r  ${epName.padEnd(30)}  ${percent.toFixed(1).padStart(5)}%`);
});
if (currentEp !== '') process.stdout.write('\n');

// Select and load a model from the catalog
const model = await manager.catalog.getModel('qwen2.5-0.5b');

await model.download((progress) => {
    process.stdout.write(`\rDownloading model: ${progress.toFixed(2)}%`);
});
console.log('\nModel downloaded.');

await model.load();
console.log('Model loaded and ready.\n');


const systemPrompt =
    'Summarize the following document into concise bullet points. ' +
    'Focus on the key points and main ideas.';

const target = process.argv[2] || 'document.txt';

try {
    const stats = statSync(target);
    if (stats.isDirectory()) {
        await summarizeDirectory(model, target, systemPrompt);
    } else {
        console.log(`--- ${basename(target)} ---`);
        const session = new ChatSession(model);
        await summarizeFile(session, target, systemPrompt);
        session.dispose();
    }
} catch {
    console.log(`--- ${basename(target)} ---`);
    const session = new ChatSession(model);
    await summarizeFile(session, target, systemPrompt);
    session.dispose();
}

// Clean up
await model.unload();
console.log('\nModel unloaded. Done!');

汇总单个文件:

node index.js document.txt

或汇总目录中的每个 .txt 文件:

node index.js ./docs

会看到类似于以下内容的输出:

Downloading model: 100.00%
Model downloaded.
Model loaded and ready.

--- document.txt ---
- Automated testing uses specialized tools to execute tests instead of manual verification.
- Tests fall into three main categories: unit tests (individual functions), integration tests
  (component interactions), and end-to-end tests (full user workflows).
- Key benefits include catching regressions early, reducing manual effort, serving as living
  documentation, and gating deployments through continuous integration pipelines.
- Effective test suites should be deterministic, independent, fast, and maintained alongside
  production code.

Model unloaded. Done!

示例存储库

本文的完整示例代码在 foundry-samples GitHub 存储库中提供。 若要克隆存储库并导航到示例,请使用:

git clone https://github.com/microsoft-foundry/foundry-samples.git
cd foundry-samples/samples/python/foundry-local/tutorial-document-summarizer

安装软件包

pip install foundry-local-sdk openai

读取文本文档

在汇总任何内容之前,需要一个示例文档来处理。 在项目目录中创建一个调用 document.txt 的文件,并添加以下内容:

Automated testing is a practice in software development where tests are written and executed
by specialized tools rather than performed manually. There are several categories of automated
tests, including unit tests, integration tests, and end-to-end tests. Unit tests verify that
individual functions or methods behave correctly in isolation. Integration tests check that
multiple components work together as expected. End-to-end tests simulate real user workflows
across the entire application.

Adopting automated testing brings measurable benefits to a development team. It catches
regressions early, before they reach production. It reduces the time spent on repetitive
manual verification after each code change. It serves as living documentation of expected
behavior, which helps new team members understand the codebase. Continuous integration
pipelines rely on automated tests to gate deployments and maintain release quality.

Effective test suites follow a few guiding principles. Tests should be deterministic, meaning
they produce the same result every time they run. Tests should be independent, so that one
failing test does not cascade into false failures elsewhere. Tests should run fast, because
slow tests discourage developers from running them frequently. Finally, tests should be
maintained alongside production code so they stay accurate as the application evolves.

现在创建一个名为 main.py 的文件,并添加以下代码以读取文档:

target = sys.argv[1] if len(sys.argv) > 1 else "document.txt"
target_path = Path(target)

该脚本接受可选的文件路径作为命令行输入参数,如果未提供,则回退到document.txt。 该方法 Path.read_text 将整个文件读入字符串。

生成摘要

初始化 Foundry Local SDK,加载模型,并发送文档内容,同时附带系统提示来让模型进行总结。

将 main.py 的内容替换为以下代码:

system_prompt = (
    "Summarize the following document into concise bullet points. "
    "Focus on the key points and main ideas."
)

target = sys.argv[1] if len(sys.argv) > 1 else "document.txt"
target_path = Path(target)

if target_path.is_dir():
    summarize_directory(model, target_path, system_prompt)
else:
    print(f"--- {target_path.name} ---")
    summarize_file(model, target_path, system_prompt)

该方法 get_model 接受模型别名,该别名是映射到目录中特定模型的短友好名称。 方法download 会将模型的权重提取到本地缓存中(如果已存在缓存会跳过下载),方法load 让模型可以进行推理准备。 系统提示要求模型生成侧重于关键想法的要点摘要。

控制摘要输出

不同的情况要求使用不同的摘要样式。 可以更改系统提示以控制模型如何构建其输出。 下面是三个有用的变体:

项目符号点(上一步的默认值):

system_prompt = (
    "Summarize the following document into concise bullet points. "
    "Focus on the key points and main ideas."
)

一段摘要:

system_prompt = (
    "Summarize the following document in a single, concise paragraph. "
    "Capture the main argument and supporting points."
)

要点:

system_prompt = (
    "Extract the three most important takeaways from the following document. "
    "Number each takeaway and keep each to one or two sentences."
)

若要尝试不同的样式,请将系统消息中的"content"值替换为其中一个提示。 该模型遵循系统提示中的说明来塑造摘要的格式和深度。

处理多个文档

将应用程序扩展以总结目录中的每个 .txt 文件。 当你有一个全部需要摘要的文档文件夹时,这非常有用。

以下函数循环访问给定目录中的所有 .txt 文件,并汇总每个文件:

async def summarize_directory(client, directory):
    txt_files = sorted(Path(directory).glob("*.txt"))

    if not txt_files:
        print(f"No .txt files found in {directory}")
        return

    for txt_file in txt_files:
        content = txt_file.read_text(encoding="utf-8")
        messages = [
            {
                "role": "system",
                "content": "Summarize the following document into concise bullet points. "
                           "Focus on the key points and main ideas."
            },
            {"role": "user", "content": content}
        ]

        print(f"--- {txt_file.name} ---")
        response = client.complete_chat(messages)
        print(response.choices[0].message.content)
        print()

每个文件都会被读取,与同一系统提示进行配对,然后独立地发送到模型。 模型不会在文件之间携带上下文,因此每个摘要都是自包含的。

完整代码

创建一个名为 main.py 并添加以下完整代码的文件:

import sys
from pathlib import Path

from foundry_local_sdk import (
    ChatSession,
    Configuration,
    FoundryLocalManager,
    MessageItem,
    Request,
    RequestOptions,
    SearchOptions,
    TextItem,
)



def get_text(response):
    parts = []
    for item in response:
        if isinstance(item, TextItem):
            parts.append(item.text)
        elif isinstance(item, MessageItem):
            parts.extend(part.text for part in item.parts if isinstance(part, TextItem))
    return "".join(parts)


def summarize_file(model, file_path, system_prompt):
    """Summarize a single file and print the result."""
    content = Path(file_path).read_text(encoding="utf-8")
    with ChatSession(model) as session:
        session.set_options(
            RequestOptions(
                search=SearchOptions(temperature=0.2, max_output_tokens=512)
            )
        )
        with (
            Request()
            .add_item(MessageItem.system(system_prompt))
            .add_item(MessageItem.user(content)) as request
        ):
            with session.process_request(request) as response:
                print(get_text(response))


def summarize_directory(model, directory, system_prompt):
    """Summarize all .txt files in a directory."""
    txt_files = sorted(Path(directory).glob("*.txt"))

    if not txt_files:
        print(f"No .txt files found in {directory}")
        return

    for txt_file in txt_files:
        print(f"--- {txt_file.name} ---")
        summarize_file(model, txt_file, system_prompt)
        print()


def main():
    # Initialize the Foundry Local SDK
    config = Configuration(app_name="foundry_local_samples")
    FoundryLocalManager.initialize(config)
    manager = FoundryLocalManager.instance

    # Discover and register execution providers that are not already available.
    missing_eps = [ep.name for ep in manager.discover_eps() if not ep.is_registered]
    if missing_eps:
        manager.download_and_register_eps(missing_eps)

    # Select and load a model from the catalog.
    model = manager.catalog.get_model("qwen2.5-0.5b")
    if model is None:
        raise RuntimeError("Model 'qwen2.5-0.5b' was not found in the catalog.")
    if not model.is_cached:
        model.download(
            lambda progress: print(
                f"\rDownloading model: {progress:.2f}%", end="", flush=True
            )
        )
        print()
    model.load()
    print("Model loaded and ready.\n")

    try:
        system_prompt = (
            "Summarize the following document into concise bullet points. "
            "Focus on the key points and main ideas."
        )

        target = sys.argv[1] if len(sys.argv) > 1 else "document.txt"
        target_path = Path(target)

        if target_path.is_dir():
            summarize_directory(model, target_path, system_prompt)
        else:
            print(f"--- {target_path.name} ---")
            summarize_file(model, target_path, system_prompt)
    finally:
        # Clean up
        if model.is_loaded:
            model.unload()
        manager.close()
        print("\nModel unloaded. Done!")


if __name__ == "__main__":
    main()

汇总单个文件:

python main.py document.txt

或汇总目录中的每个 .txt 文件:

python main.py ./docs

会看到类似于以下内容的输出:

Downloading model: 100.00%
Model loaded and ready.

--- document.txt ---
- Automated testing uses specialized tools to execute tests instead of manual verification.
- Tests fall into three main categories: unit tests (individual functions), integration tests
  (component interactions), and end-to-end tests (full user workflows).
- Key benefits include catching regressions early, reducing manual effort, serving as living
  documentation, and gating deployments through continuous integration pipelines.
- Effective test suites should be deterministic, independent, fast, and maintained alongside
  production code.

Model unloaded. Done!

示例存储库

本文的完整示例代码在 foundry-samples GitHub 存储库中提供。 若要克隆存储库并导航到示例,请使用:

git clone https://github.com/microsoft-foundry/foundry-samples.git
cd foundry-samples/samples/rust/foundry-local/tutorial-document-summarizer

安装软件包

cargo add foundry-local-sdk
cargo add tokio --features full
cargo add tokio-stream anyhow

读取文本文档

在汇总任何内容之前,需要一个示例文档来处理。 在项目目录中创建一个调用 document.txt 的文件,并添加以下内容:

Automated testing is a practice in software development where tests are written and executed
by specialized tools rather than performed manually. There are several categories of automated
tests, including unit tests, integration tests, and end-to-end tests. Unit tests verify that
individual functions or methods behave correctly in isolation. Integration tests check that
multiple components work together as expected. End-to-end tests simulate real user workflows
across the entire application.

Adopting automated testing brings measurable benefits to a development team. It catches
regressions early, before they reach production. It reduces the time spent on repetitive
manual verification after each code change. It serves as living documentation of expected
behavior, which helps new team members understand the codebase. Continuous integration
pipelines rely on automated tests to gate deployments and maintain release quality.

Effective test suites follow a few guiding principles. Tests should be deterministic, meaning
they produce the same result every time they run. Tests should be independent, so that one
failing test does not cascade into false failures elsewhere. Tests should run fast, because
slow tests discourage developers from running them frequently. Finally, tests should be
maintained alongside production code so they stay accurate as the application evolves.

现在打开 src/main.rs 并添加以下代码以读取文档:

let target = env::args()
    .nth(1)
    .unwrap_or_else(|| "document.txt".to_string());
let target_path = Path::new(&target);

代码接受一个可选的文件路径作为命令行参数,如果未提供路径,则回退到document.txt。

生成摘要

初始化 Foundry Local SDK,加载模型,并发送文档内容,同时附带系统提示来让模型进行总结。

将 src/main.rs 的内容替换为以下代码:

let system_prompt = "Summarize the following document into concise bullet points. Focus on \
                     the key points and main ideas.";

let target = env::args()
    .nth(1)
    .unwrap_or_else(|| "document.txt".to_string());
let target_path = Path::new(&target);

if target_path.is_dir() {
    summarize_directory(&model, target_path, system_prompt).await?;
} else {
    let file_name = target_path
        .file_name()
        .map(|name| name.to_string_lossy().to_string())
        .unwrap_or_else(|| target.clone());
    println!("--- {file_name} ---");
    summarize_file(&model, target_path, system_prompt).await?;
}

该方法 get_model 接受模型别名,该别名是映射到目录中特定模型的短友好名称。 方法download 会将模型的权重提取到本地缓存中(如果已存在缓存会跳过下载),方法load 让模型可以进行推理准备。 系统提示要求模型生成侧重于关键想法的要点摘要。

控制摘要输出

不同的情况要求使用不同的摘要样式。 可以更改系统提示以控制模型如何构建其输出。 下面是三个有用的变体:

项目符号点(上一步的默认值):

let system_prompt =
    "Summarize the following document into concise bullet points. \
     Focus on the key points and main ideas.";

一段摘要:

let system_prompt =
    "Summarize the following document in a single, concise paragraph. \
     Capture the main argument and supporting points.";

要点:

let system_prompt =
    "Extract the three most important takeaways from the following document. \
     Number each takeaway and keep each to one or two sentences.";

若要尝试其他样式,请将系统消息内容替换为其中一个提示。 该模型遵循系统提示中的说明来塑造摘要的格式和深度。

处理多个文档

将应用程序扩展以总结目录中的每个 .txt 文件。 当你有一个全部需要摘要的文档文件夹时,这非常有用。

以下函数循环访问给定目录中的所有 .txt 文件,并汇总每个文件:

use std::path::Path;

async fn summarize_directory(
    client: &foundry_local_sdk::ChatClient,
    directory: &Path,
    system_prompt: &str,
) -> anyhow::Result<()> {
    let mut txt_files: Vec<_> = fs::read_dir(directory)?
        .filter_map(|entry| entry.ok())
        .filter(|entry| {
            entry.path().extension()
                .map(|ext| ext == "txt")
                .unwrap_or(false)
        })
        .collect();

    txt_files.sort_by_key(|e| e.path());

    if txt_files.is_empty() {
        println!("No .txt files found in {}", directory.display());
        return Ok(());
    }

    for entry in &txt_files {
        let file_content = fs::read_to_string(entry.path())?;
        let messages: Vec<ChatCompletionRequestMessage> = vec![
            ChatCompletionRequestSystemMessage::new(system_prompt).into(),
            ChatCompletionRequestUserMessage::new(&file_content).into(),
        ];

        let file_name = entry.file_name();
        println!("--- {} ---", file_name.to_string_lossy());
        let resp = client.complete_chat(&messages, None).await?;
        let text = resp.choices[0]
            .message
            .content
            .as_deref()
            .unwrap_or("");
        println!("{}\n", text);
    }

    Ok(())
}

每个文件都会被读取,与同一系统提示进行配对,然后独立地发送到模型。 模型不会在文件之间携带上下文,因此每个摘要都是自包含的。

完整代码

将 src/main.rs 的内容替换为完整的代码:

use foundry_local_sdk::{
    ChatSession, FoundryLocalConfig, FoundryLocalManager, Item, Model, Request, RequestOptions,
    SearchOptions,
};
use std::io::{self, Write};
use std::path::Path;
use std::{env, fs};

async fn summarize_file(
    model: &Model,
    file_path: &Path,
    system_prompt: &str,
) -> Result<(), Box<dyn std::error::Error>> {
    let content = fs::read_to_string(file_path)?;
    let session = ChatSession::new(model).await?;
    session
        .set_options(RequestOptions {
            search: SearchOptions {
                temperature: Some(0.7),
                max_output_tokens: Some(512),
                ..Default::default()
            },
            ..Default::default()
        })
        .await?;

    let response = session
        .process_request(Request::from_items(vec![
            Item::system_message(vec![Item::text(system_prompt)]),
            Item::user_message(vec![Item::text(content)]),
        ]))
        .await?;

    for item in &response.items {
        if let Some(message) = item.as_message() {
            print!("{}", message.text());
        } else if let Some(text) = item.as_text() {
            print!("{text}");
        }
    }
    println!();
    drop(session);
    Ok(())
}

async fn summarize_directory(
    model: &Model,
    directory: &Path,
    system_prompt: &str,
) -> Result<(), Box<dyn std::error::Error>> {
    let mut txt_files: Vec<_> = fs::read_dir(directory)?
        .filter_map(Result::ok)
        .filter(|entry| entry.path().extension().is_some_and(|ext| ext == "txt"))
        .collect();

    txt_files.sort_by_key(|entry| entry.path());

    if txt_files.is_empty() {
        println!("No .txt files found in {}", directory.display());
        return Ok(());
    }

    for entry in &txt_files {
        println!("--- {} ---", entry.file_name().to_string_lossy());
        summarize_file(model, &entry.path(), system_prompt).await?;
        println!();
    }

    Ok(())
}

#[tokio::main]
async fn main() -> Result<(), Box<dyn std::error::Error>> {
    // Initialize the Foundry Local SDK
    let manager = FoundryLocalManager::create(FoundryLocalConfig::new("foundry_local_samples"))?;

    manager
        .download_and_register_eps_with_progress(None, {
            let mut current_ep = String::new();
            move |ep_name: &str, percent: f64| {
                if ep_name != current_ep {
                    if !current_ep.is_empty() {
                        println!();
                    }
                    current_ep = ep_name.to_string();
                }
                print!("\r  {ep_name:<30}  {percent:5.1}%");
                io::stdout().flush().ok();
            }
        })
        .await?;
    println!();

    let model = manager.catalog().get_model("qwen2.5-0.5b").await?;
    if !model.is_cached().await? {
        println!("Downloading model...");
        model
            .download(Some(|progress: f64| {
                print!("\r  {progress:.1}%");
                io::stdout().flush().ok();
            }))
            .await?;
        println!();
    }

    model.load().await?;
    println!("Model loaded and ready.\n");

    let system_prompt = "Summarize the following document into concise bullet points. Focus on \
                         the key points and main ideas.";

    let target = env::args()
        .nth(1)
        .unwrap_or_else(|| "document.txt".to_string());
    let target_path = Path::new(&target);

    if target_path.is_dir() {
        summarize_directory(&model, target_path, system_prompt).await?;
    } else {
        let file_name = target_path
            .file_name()
            .map(|name| name.to_string_lossy().to_string())
            .unwrap_or_else(|| target.clone());
        println!("--- {file_name} ---");
        summarize_file(&model, target_path, system_prompt).await?;
    }

    model.unload().await?;
    println!("\nModel unloaded. Done!");

    Ok(())
}

汇总单个文件:

cargo run -- document.txt

或汇总目录中的每个 .txt 文件:

cargo run -- ./docs

会看到类似于以下内容的输出:

Downloading model: 100.00%
Model loaded and ready.

--- document.txt ---
- Automated testing uses specialized tools to execute tests instead of manual verification.
- Tests fall into three main categories: unit tests (individual functions), integration tests
  (component interactions), and end-to-end tests (full user workflows).
- Key benefits include catching regressions early, reducing manual effort, serving as living
  documentation, and gating deployments through continuous integration pipelines.
- Effective test suites should be deterministic, independent, fast, and maintained alongside
  production code.

Model unloaded. Done!

示例存储库

克隆 Foundry 示例GitHub存储库并转到 C++ 示例:

git clone https://github.com/microsoft-foundry/foundry-samples.git
cd foundry-samples/samples/cpp/foundry-local/tutorial-document-summarizer

安装软件包

C++ 示例需要:

  • CMake 3.20 或更高版本。
  • C++20 编译器。
  • PATH 上的 NuGet CLI。

配置并生成示例:

cmake -S . -B build
cmake --build build --config Release

在配置期间,示例将版本 2.2.1 的 Microsoft.AI.Foundry.Local.Runtime NuGet 包还原到 build/_packages。 该包包含用于 Windows、Linux 和 macOS 的 C++ 标头和本机库。 不需要单独的 WinML 包。

CMake 配置会为您的操作系统和体系结构选择运行时资源,并将所需的本机库复制到可执行文件所在目录。 在 macOS 和 Linux 上,从 build中运行可执行文件。 在 Windows 上,运行位于 .exe 的 build 文件。

读取文本文档

从命令行路径读取文本文件:

const std::string path = argc > 1 ? argv[1] : "document.txt";
std::ifstream input(path);
if (!input) {
  throw std::runtime_error("Could not open document: " + path);
}
const std::string document{
    std::istreambuf_iterator<char>(input),
    std::istreambuf_iterator<char>(),
};
if (document.empty()) {
  throw std::runtime_error("The document is empty.");
}

生成摘要

加载聊天模型并提示它汇总文档:

{
  ChatSession session(*model);
  Request request{
      SystemMessage(
          "Summarize the supplied document using a short overview and three bullet points."),
      UserMessage(document),
  };
  request.SetOptions({
      .search = {.temperature = 0.2F, .max_output_tokens = 512, .seed = 42},
  });
  std::cout << ResponseText(session.ProcessRequest(request)) << "\n";
}

完整代码

#include <foundry_local/foundry_local_cpp.h>

#include <fstream>
#include <iostream>
#include <iterator>
#include <string>

using namespace foundry_local;

std::string ResponseText(const Response& response) {
  std::string text;
  for (const Item& item : response.GetItems()) {
    if (item.GetType() != FOUNDRY_LOCAL_ITEM_MESSAGE) {
      continue;
    }
    for (const Item& part : item.GetMessage().parts) {
      if (part.GetType() == FOUNDRY_LOCAL_ITEM_TEXT) {
        text.append(part.GetText().text);
      }
    }
  }
  return text;
}

int main(int argc, char* argv[]) {
  try {
    Configuration config("foundry_local_cpp_document_summarizer");
    Manager manager(std::move(config));
    auto model = manager.GetCatalog().GetModel("qwen2.5-0.5b");
    if (!model) {
      throw std::runtime_error("Model 'qwen2.5-0.5b' was not found in the catalog.");
    }
    if (!model->IsCached()) {
      model->Download([](float progress) {
        std::cout << "\rDownloading model: " << static_cast<int>(progress) << "%" << std::flush;
        return 0;
      });
      std::cout << "\n";
    }
    model->Load();

    const std::string path = argc > 1 ? argv[1] : "document.txt";
    std::ifstream input(path);
    if (!input) {
      throw std::runtime_error("Could not open document: " + path);
    }
    const std::string document{
        std::istreambuf_iterator<char>(input),
        std::istreambuf_iterator<char>(),
    };
    if (document.empty()) {
      throw std::runtime_error("The document is empty.");
    }

    {
      ChatSession session(*model);
      Request request{
          SystemMessage(
              "Summarize the supplied document using a short overview and three bullet points."),
          UserMessage(document),
      };
      request.SetOptions({
          .search = {.temperature = 0.2F, .max_output_tokens = 512, .seed = 42},
      });
      std::cout << ResponseText(session.ProcessRequest(request)) << "\n";
    }

    model->Unload();
    manager.Shutdown();
  } catch (const Error& error) {
    std::cerr << "Foundry Local error [" << error.Code() << "]: " << error.what() << "\n";
    return 1;
  } catch (const std::exception& error) {
    std::cerr << "Error: " << error.what() << "\n";
    return 1;
  }

  return 0;
}

总结文件:

./build/tutorial-document-summarizer document.txt

清理资源

卸载模型后,模型权重将保留在本地缓存中。 这意味着下次运行应用程序时,将跳过下载步骤,模型加载速度更快。 除非需要回收磁盘空间,否则不需要进行额外的清理。