Automated Build Failure Analysis with Log Detective in Packit

Overview

If you've ever faced a cryptic package build failure in Fedora's Koji build system, you know the frustration of sifting through thousands of lines of logs. Log Detective, now integrated with Packit, offers a helping hand. It automatically analyzes failed Koji scratch builds triggered by Packit on dist-git pull requests, providing a human-readable explanation and even a suggested fix. This tutorial walks you through what Log Detective does, how to take advantage of it, and what to watch out for—even though it requires zero configuration on your part.

Automated Build Failure Analysis with Log Detective in Packit — Source: fedoramagazine.org

Prerequisites

Before diving in, ensure you have the following in place:

A Fedora dist-git repository with Packit integration – Your project must be connected to Packit, which automates builds on Koji when you create a pull request.
Access to the Packit Dashboard – Log Detective results are displayed there. You'll need a Fedora account to log in.
A general understanding of Packit workflows (optional) – Knowing how Packit triggers scratch builds helps, but isn't required to benefit from the analysis.
No separate setup needed – Log Detective is enabled by default for all Packit-triggered Koji scratch builds. You don't need to install anything or configure prompts.

Step-by-Step: How a Build Failure Triggers Log Detective

The entire process is automatic. Here’s what happens behind the scenes and what you’ll see as a user.

1. Configure Packit (If Not Already Done)

Although no special Log Detective configuration is needed, you must have a working packit.yaml file in your dist-git repository that enables Koji scratch builds. Below is a minimal example.

# packit.yaml
specfile: mypackage.spec
jobs:
  - job: scratch_build
    trigger: pull_request
    metadata:
      targets:
        - fedora-all

This tells Packit to perform a scratch Koji build for every pull request. If the build fails, Log Detective will automatically analyze the logs.

2. Submit a Pull Request That Causes a Build Failure

Create a pull request in your dist-git repository. For this tutorial, introduce an intentional error in the spec file, e.g., a missing BuildRequires or a syntax error. Once submitted, Packit will trigger a scratch build on Koji.

Watch the Packit Dashboard for the build status. Initially you'll see "running" or "pending".

3. Automatic Analysis Request

When the build fails, Packit automatically sends a request to the Log Detective interface server. This server is a lightweight, containerized service that orchestrates the analysis.

What is sent: All build logs (e.g., build.log, root.log, state.log) and other artifacts. You don't need to choose which files or craft a prompt—the service does everything.
How the analysis works internally: Log Detective (version 4.0+) uses an agent built on the BeeAI Framework. It employs a variety of tools, notably the Drain template mining algorithm, to extract meaningful snippets from the logs. Snippets are small fractions of the original files, saving tokens and reducing processing time while keeping relevant information.

4. Retrieving and Displaying Results

Once the analysis finishes, the interface server posts the results on the Fedora Messaging bus. Packit listens for these messages and links the analysis to the corresponding pull request in the Dashboard.

View the result: Go to the Packit Dashboard, find your PR, and click on the build result. Look for a link or section labeled Log Detective Analysis. It will show:

What went wrong: A statement identifying the root cause (e.g., "Missing dependency: python3-devel is not installed").
Suggested fix: (optional) A recommendation, such as "Add BuildRequires: python3-devel to the spec file."

The analysis is derived solely from the build logs; Log Detective does not consult external sources like bug trackers or package metadata.

Code Example: Example Analysis Response (JSON)

While you won't see raw JSON in the Dashboard, the internal API returns something like this. This is for illustration only.

{
  "analysis_id": "abc123",
  "status": "completed",
  "statement": "Build failed because the required tool 'cmake' is not found in the build environment.",
  "suggestion": "Ensure 'cmake' is listed in BuildRequires in the spec file."
}

You'll see a human-readable version of this on your PR page.

Common Mistakes & Misconceptions

Assuming Log Detective works for every failure. The tool uses a general-purpose language model and the Drain algorithm; it may not always pinpoint the exact issue, especially with rare or subtle errors. It's an assistant, not a replacement for debugging skills.
Expecting results immediately. Analysis takes a few seconds to a minute depending on log size. If you don't see it, refresh the dashboard, but don't worry—it's usually ready within a minute of the build failure.
Trying to manually trigger or customize the analysis. In Packit, the trigger is automatic and cannot be turned off or on. If you want to analyze a different build (e.g., a successful one, or one from Copr), use the "Ask AI" button in the Copr interface—but that's separate from the Packit integration.
Thinking it replaces experience. As noted, Log Detective is aimed at newcomers to Fedora packaging. Veteran packagers will likely find it less useful because they already know common failure patterns. Don't ignore your own intuition.
Overlooking the need for valid Packit configuration. If Packit is not set up for Koji scratch builds, Log Detective won't even get a chance to run. Double-check your packit.yaml.

Future Development

The Log Detective service is under active development. Planned improvements include:

Integration with additional data sources beyond build logs (e.g., package changelogs, Copr repositories).
Better handling of multi-line error messages and cross-file dependencies.
Possibility to run analysis on-demand via a web interface instead of only on failure.

Stay tuned to the Packit project for updates.

Summary

Log Detective in Packit eliminates the tedious log-diving that follows a failed Koji scratch build. By automatically extracting informative snippets using the Drain algorithm and an AI agent, it provides a concise explanation and a fix suggestion—all without any manual effort. While it's not perfect and won't replace deep packaging expertise, it's a powerful aid for both novices and busy maintainers. Just ensure your Packit integration is active, and let Log Detective do the heavy lifting.

Tags: