Why Kmpkg
Why design and develop kmpkg
1. The problem: C++ package management breaks down in production
C++ package management problems do not primarily come from missing features. They come from production reality:
- builds must be reproducible across machines and time
- binaries must be reused instead of rebuilt
- networks may be restricted or completely offline
- deployment environments differ from development environments
- build systems and toolchains are heterogeneous and long-lived
Most existing C/C++ package managers work reasonably well for development, but start to fail when pushed into long-term, production-scale usage.
kmpkg was designed specifically to address this gap.
2. Lessons learned from existing tools
Before kmpkg, multiple package managers were evaluated and used in real projects:
- cget: simple, but too limited for complex dependency graphs
- Conan: powerful, but configuration-heavy and difficult to reason about at scale
- Conda: effective for Python-centric ecosystems, but mismatched with C++ build realities
- vcpkg: strong foundation, but tightly coupled to centralized assumptions and source-first workflows
Each of these tools solves part of the problem, but none form a clean, deterministic, and production-friendly system when binaries, mirrors, and deployment constraints become first-class concerns.
3. Binary-first thinking comes from real cost
In large C++ systems:
- rebuilds are expensive
- CI time dominates iteration speed
- rebuilding the same dependency repeatedly is wasteful
- “source-only” workflows do not scale operationally
Before kmpkg, an internal tool called carbin was developed to experiment with binary-centric dependency workflows. This experiment made one thing clear:
Binary reuse is not an optimization — it is a requirement.
kmpkg inherits this lesson directly.
4. Mirrors must be first-class, not an afterthought
Most package managers treat mirrors as:
- optional
- secondary
- or merely caching layers
In production environments, mirrors are none of those things.
They are essential for:
- network isolation
- compliance and auditing
- deployment determinism
- disaster recovery
- multi-region consistency
kmpkg elevates mirror and distribution control to top-level concepts. Mirrors are explicit, configurable, and central to how the system operates.
This design makes kmpkg naturally suitable for:
- private infrastructures
- restricted or air-gapped networks
- enterprise production deployments
5. Inspiration, not imitation
kmpkg is inspired by:
- vcpkg: clear package model and CMake integration
- Go modules: minimal configuration and deterministic resolution
- Rust Cargo: unified workflow and strong version discipline
However, kmpkg does not attempt to copy these systems.
C++ has unique constraints:
- multiple build systems
- ABI and toolchain fragmentation
- long-lived dependencies
- heavy compile cost
kmpkg is designed around these realities instead of abstracting them away.
6. kmpkg is part of a toolchain, not a standalone tool
A package manager alone cannot solve C++ productivity problems.
kmpkg is designed to integrate naturally with surrounding tools:
- kmcmake for project and build generation
- kmdo for operational workflows such as cloning, merging, and publishing mirrors and packages
Together, these tools form a coherent system where:
- dependency management
- build configuration
- binary distribution
- release and deployment
are treated as parts of the same workflow, not disconnected steps.
7. Design goals
The core design goals of kmpkg are:
- determinism: the same inputs produce the same outputs
- operability: easy to reason about, debug, and operate
- binary reuse: avoid unnecessary rebuilds
- mirror-first: distribution control is explicit and central
- toolchain integration: fit naturally into real C++ workflows
kmpkg deliberately avoids chasing feature completeness at the cost of these principles.
8. What kmpkg is not trying to be
kmpkg is not trying to:
- replace all existing C++ tools
- abstract away CMake, compilers, or linkers
- hide complexity through opaque magic
- optimize for quick demos over long-term operation
It is designed for engineers who care about systems that survive years of production use.
Demand-Driven, Practice-Driven Design
kmpkg was not designed by listing features. It was shaped by real production failures, operational constraints, and long-term maintenance cost.
At the system design level, kmpkg follows one strict rule:
The user is the only first-class citizen. The system is a tool, not a policy.
This principle is non-negotiable.
Most package managers gradually evolve toward protecting their own internal consistency: global registries, centralized assumptions, hidden defaults, and “recommended” workflows that quietly become mandatory. Over time, the system starts to optimize for itself instead of the engineers operating it.
kmpkg explicitly rejects this direction.
System design in kmpkg is demand-driven: every abstraction exists only because a real user needed it to solve a real problem in production. If a feature cannot be justified by practical usage, it does not belong in the system.
The system never assumes authority over the user’s environment.
- It does not assume network accessibility.
- It does not assume uniform toolchains.
- It does not assume binary compatibility.
- It does not assume a single “correct” way to build or deploy.
Instead, kmpkg exposes mechanisms and keeps policy external. Users decide where artifacts come from, how they are built, which binaries are acceptable, and when reuse is safe.
This design philosophy directly influences every major system decision, including binary reuse.
A Concrete Example: RocksDB
Consider RocksDB.
In real-world production, RocksDB is almost never consumed as a single, universal binary. Compression support alone already fragments the space: lz4, zstd, snappy, or none. Add RTTI, exception handling, platform-specific toolchains, and the idea of a “standard binary” collapses immediately.
At this point, pure binary distribution stops being a solution.
Binary repositories grow exponentially, yet still fail to provide the correct artifact. Clients cannot reliably select the right binary, because compatibility depends on build-time decisions invisible at the package boundary.
The result is predictable:
- Engineers repeatedly rebuild the same dependency with slightly different flags
- Projects behave differently across machines
- Configuration drifts silently over time
- Reproducibility degrades into tribal knowledge
This is not a tooling mistake. It is a system design failure — one that prioritizes the package manager’s abstraction over the user’s reality.
kmpkg takes the opposite stance.
It accepts that libraries like RocksDB cannot be normalized away. Build variability is explicit. Distribution control is user-owned. Binary reuse is allowed only where it is operationally safe — never assumed by default.
The system adapts to the user, not the other way around.
System as Infrastructure, Not Authority
kmpkg does not attempt to “standardize” C++ development. C++ does not need another authority.
Instead, kmpkg behaves like infrastructure:
- predictable
- inspectable
- replaceable
- and silent when not needed
The moment a system forces users to change how they work to satisfy its internal model, it has failed its purpose.
kmpkg exists to reduce friction, not to define correctness.
Compare Concepts of Package Managers
Different C/C++ package managers often appear to solve the same problem, but they are built on fundamentally different assumptions. These assumptions determine whether a system survives real production environments or collapses under operational pressure.
This section compares major package managers at the conceptual level, not feature lists.
Core Design Axes
For production C++ systems, the following axes dominate all real-world outcomes:
- Who owns build decisions
- How binaries are treated
- Where control boundaries lie
- Whether the system optimizes for itself or for users
cget
Conceptual model: Minimal source-based dependency fetcher.
Key assumptions:
- Build happens locally
- Users fully control compilation
- No global dependency graph
- No binary reuse model
Strengths:
- Simple
- Transparent
- Low abstraction cost
Structural limitations:
- No system-level coordination
- No reproducibility guarantees across machines
- No production-scale distribution model
cget works well as a mechanical helper, but does not attempt to be an ecosystem. It scales poorly beyond individual projects.
Conan
Conceptual model: Centralized package manager with configuration-driven binaries.
Key assumptions:
- Binaries can be parameterized sufficiently via settings/options
- Central registries are acceptable
- Dependency graphs can be globally resolved
Strengths:
- Rich metadata model
- Explicit configuration space
- Strong focus on binary reuse
Structural limitations:
- Configuration explosion in real-world libraries
- Binary compatibility becomes combinatorial
- Central authority gradually dictates workflows
- Operational complexity leaks to users
In practice, Conan often shifts complexity from builds into metadata and infrastructure. The system grows more complex as variability increases.
Conda
Conceptual model: Binary-first, environment-isolated distribution system.
Key assumptions:
- Prebuilt binaries are the primary artifact
- Environments are isolated and disposable
- Toolchains are controlled centrally
Strengths:
- Extremely strong reproducibility
- Works well for homogeneous stacks (Python, data science)
Structural limitations (for C++):
- Binary rigidity
- Poor fit for heterogeneous toolchains
- Limited adaptability to custom build flags
- Hard boundaries between environments and native systems
Conda succeeds by restricting variability, which conflicts directly with most large-scale C++ production environments.
vcpkg
Conceptual model: Source-based ports with optional binary caching.
Key assumptions:
- Source builds are the safest baseline
- A shared ports registry is acceptable
- Binary reuse is a performance optimization, not a requirement
Strengths:
- Strong CMake integration
- Predictable source builds
- Relatively simple mental model
Structural limitations:
- Registry-centric worldview
- Mirrors treated as secondary
- Operational control remains implicit
- Limited expression of distribution policy
vcpkg works well for many development scenarios, but its architecture assumes a benign, globally connected environment.
kmpkg
Conceptual model: User-controlled distribution and build infrastructure.
Key assumptions:
- Users operate under diverse constraints
- Mirrors and distribution are first-class
- Binary reuse must be explicit and conditional
- The system must never override user intent
Key distinctions:
- Mirrors are top-level concepts, not implementation details
- No assumption of global registries
- Build variability is expected, not normalized away
- Mechanism is preferred over policy
kmpkg is not designed to define “the correct way” to build C++. It is designed to survive long-term production use, where requirements evolve and constraints accumulate.
Conceptual Summary
| Dimension | cget | Conan | Conda | vcpkg | kmpkg |
|---|---|---|---|---|---|
| Binary-first | No | Yes | Yes | Optional | Conditional |
| Source-first | Yes | Partial | No | Yes | Yes |
| Central registry | No | Yes | Yes | Yes | No |
| Mirror as first-class | No | No | Partial | No | Yes |
| Build variability tolerance | High | Medium | Low | Medium | High |
| User as super citizen | Partial | No | No | Partial | Yes |
Why This Matters
Most failures in C++ package management are not tooling bugs. They are the inevitable result of mismatched assumptions.
kmpkg is designed around one reality:
Production systems change faster than package managers.
Any system that prioritizes internal consistency over user control will eventually fail under real operational pressure.
A Real Production Failure Cases
vcpkg
A common failure mode appeared again when recommending kmpkg to external teams.
The team was using vcpkg in production. In order to keep builds stable, they had no choice but to pin the vcpkg repository version and follow the official registry strictly.
This worked—until it didn’t.
When the upstream vcpkg registry evolved, their project stopped compiling. The failure was not caused by code changes in their own system, but by changes in dependency build logic and port definitions upstream.
At that point, recovery became manual and expensive:
- They had to search Git history in the vcpkg repository
- Identify which commit was previously used
- Manually check out that historical state
- Hope that it still matched their toolchain and environment
This process was:
- Slow
- Error-prone
- Impossible to automate reliably
- Completely disconnected from the project’s own versioning
Most importantly, the dependency system—not the user—controlled the upgrade path.
The Structural Problem
This is not a vcpkg bug. It is a design consequence.
When a package manager:
- Treats the official registry as authoritative
- Treats mirrors as secondary
- Couples build logic tightly to upstream evolution
then users lose operational control.
Version pinning becomes a defensive hack, not a first-class concept. Rollback becomes archaeology.
kmpkg’s Design Response
kmpkg was designed specifically to avoid this failure mode.
- Mirrors are first-class and user-owned
- Distribution state is explicit, not implicit
- Upstream evolution is opt-in, not forced
- Historical states are preserved as part of the workflow, not Git accidents
In kmpkg, the user decides when and how the ecosystem moves. The system never assumes that upstream knows better than production reality.
The Principle
In production, stability is not achieved by freezing the world. It is achieved by owning the world you depend on.
kmpkg exists because too many C/C++ teams learned this lesson the hard way.
Why Registry-Centric Design Fails at Scale
Traditional C/C++ package managers are heavily registry-centric. They assume that upstream knows the “correct” configuration, binary layout, and dependency graph. This assumption works for simple projects, but breaks at scale.
Example: SIMD-dependent libraries. Many high-performance libraries (like compression engines, linear algebra, or databases) provide multiple build variants:
- SIMD instruction sets: AVX2, SSE4, NEON
- Optional features: LZ4, ZSTD, Snappy support
- RTTI, exceptions, or ABI differences
A registry-centric approach often provides one canonical binary per version, assuming the build options are universal. In reality:
- A client may run on different machines with different CPU capabilities.
- One library may need different compression or SIMD flags depending on the subsystem.
- RTTI or ABI mismatches can explode the binary repository, because every small combination multiplies the number of binaries required.
In practice, this leads to:
- Registry explosion: The central repo grows combinatorially with every optional feature.
- Binary incompatibility: Clients can’t pick the right prebuilt binary for their exact environment.
- Operational friction: Teams are forced to rebuild binaries locally or patch the registry, violating reproducibility and automation.
Even worse, the user loses control:
- Users cannot declaratively select the exact variant they need per machine or per subsystem.
- Rollback to a previous state may be impossible without forking the registry or rebuilding dozens of combinations manually.
- Production stability is hostage to upstream decisions.
kmpkg’s Response
kmpkg flips this model:
- User-centric: The user chooses the mirror, the binary variants, and the deployment strategy.
- Binary reuse: The system supports multiple variants explicitly, without registry explosion.
- Reproducible workflows: Each project has full control over which binaries are used, even on heterogeneous machines.
- Feature flexibility: Optional features (like SIMD or compression backends) are first-class, combinable locally without central coordination.
Principle: In real production environments, the registry is a tool, not the boss. Control must reside with the user, not the package manager.
Sure. Here's a unified English version of the kmpkg Core Design points with concise, practice-driven emphasis:
kmpkg Core Design
-
Clear Dependency Configuration All project dependencies are explicitly declared and versioned, making it obvious what each project depends on.
-
Feature-Aware Dependencies Dependencies support feature flags and transitive requirements, allowing fine-grained control over which capabilities are included in the build.
-
Mirror-First Architecture Mirrors are treated as first-class entities, enabling reliable operation in private networks, air-gapped environments, or restricted infrastructures.
-
Flexible Binary Variants Binary artifacts can vary according to compiler, platform, or feature set. This prevents binary explosion while supporting reproducible builds across heterogeneous environments.
-
User as Super-Citizen Users control the workflow; the system acts purely as a tool. This ensures operational decisions always favor the needs of the developer or project team.
-
Operational Simplicity Designed to minimize configuration overhead, reduce friction in everyday operations, and ensure reproducibility without complex manual intervention.
-
Integration with Toolchain Seamless integration with existing C++ ecosystem tools such as CMake, MSBuild, and internal tools like kmcmake and kmdo. Project generation, dependency management, build configuration, and release form a unified workflow while maintaining compatibility with standard C++ engineering practices.