Information Cost of Concept Completion

Abstract

Completion cost is conditional entropy on supported concept fibers, not a global factorization criterion.

Theorem 1.1 (Completion cost is conditional entropy).

$$\begin{array}{c}\forall X,C,K[\mathrm{Fintype}(X)][\mathrm{Fintype}(C)][\mathrm{Fintype}(K)],\\ mass:X\to \mathbb{R},concept:X\to C,target:X\to K,\\ (\forall x,0\le mass(x))\wedge \sum _{x}mass(x)=1\Rightarrow \\ \mathrm{shannonEntropy}(\mathrm{completionLaw}\left(mass,concept,target\right))-\mathrm{shannonEntropy}(\mathrm{pushforward}\left(concept,mass\right))=\mathrm{conditionalEntropy}(\mathrm{completionLaw}\left(mass,concept,target\right)).\end{array}$$

Proof. Machine-checked in Lean as D5/S3/ConceptDynamics/Completion/CompletionInformationCost.completion_information_cost (✓ std3). ∎

Source. Repository-derived.

Commentary.

Let a normalized nonnegative mass be given on a finite source, with a concept readout and a target readout. Their completion law is the joint distribution obtained by sending each source point to its pair of readout values.

The first marginal of this joint law is exactly the distribution of the concept readout. The finite entropy chain rule therefore identifies the entropy gained by adjoining the target coordinate with the target’s entropy conditional on the current concept.

Only concept fibers carrying positive mass contribute to this cost. No strict positivity assumption is imposed on individual source points or concept fibers.

Lemma 1.2 (Zero completion cost need not give a global target factor).

$$\begin{array}{c}\exists mass:\mathrm{Fin}\left(3\right)\to \mathbb{R},concept,target:\mathrm{Fin}\left(3\right)\to Bool,\\ (\forall x,0\le mass(x))\wedge \sum _{x}mass(x)=1\wedge \\ \mathrm{conditionalEntropy}(\mathrm{completionLaw}\left(mass,concept,target\right))=0\wedge \\ (\neg \exists factor:Bool\to Bool,target=factor\circ concept)\wedge \\ \exists x,y:\mathrm{Fin}\left(3\right),mass(x)=0\wedge mass(y)=0\wedge \\ concept(x)=concept(y)\wedge target(x)\ne target(y).\end{array}$$

Proof. Machine-checked in Lean as D5/S3/ConceptDynamics/Completion/CompletionInformationCost.zero_conditional_entropy_not_global_factorization (✓ std3). ∎

Source. Repository-derived.

Commentary.

On a three-point source, put all mass on the first point, use the constant false concept readout, and let the target be false on the first two points and true on the third. The only supported conditional slice is a point mass, so its conditional entropy is zero.

The two zero-mass points have the same concept value but different target values. Consequently no Boolean function can recover the target from the concept on the whole source type. Zero completion cost therefore controls supported fibers only, not unsupported points.

References

• Truth anchor: D5/S3/ConceptDynamics/Completion/CompletionInformationCost.completion_information_cost
• Truth anchor: D5/S3/ConceptDynamics/Completion/CompletionInformationCost.zero_conditional_entropy_not_global_factorization
• Dependency: D5/S3/Entropy/ConditionalEntropyEquality
• Dependency: D5/S3/Entropy/Forgetting/CapacityMonotone