metadb: add optional fast restart path that skips storage walk when (version + commit + storage config) matches metaDB stamp (#4026)

* chore(metadb): add writer version to interface

Signed-off-by: Jacob McSwain <jacob@mcswain.dev>

* chore(metadb): add writer version to db mock

Signed-off-by: Jacob McSwain <jacob@mcswain.dev>

* chore(metadb): implement writer version for bolt, redis, and dynamodb

Signed-off-by: Jacob McSwain <jacob@mcswain.dev>

* feat(metadb): add optional fast restart path that skips storage walk when binary identity matches metaDB stamp

binary identity is determined by the current release tag/commit and stored in metaDB after a successful storage parse. When fast restart is enabled, the next startup will skip the parse if the stored identity matches the current binary

Signed-off-by: Jacob McSwain <jacob@mcswain.dev>

* chore(cli): serve: add a way to force reparse storage

Signed-off-by: Jacob McSwain <jacob@mcswain.dev>

* refactor(meta): version: split to avoid global state mutation in tests

Signed-off-by: Jacob McSwain <jacob@mcswain.dev>

* fix(meta): version: include commit in writerVersion to distinguish retags

Signed-off-by: Jacob McSwain <jacob@mcswain.dev>

* chore(config): add IsFastRestartEnabled() test

Signed-off-by: Jacob McSwain <jacob@mcswain.dev>

* fix(meta): skip writer-version stamp when storage parse is incomplete

ParseStorage returns nil even when individual repos fail to parse or are only partially parsed (a missing manifest blob), so MaybeParseStorage would stamp a partially-populated metaDB as good. On the next restart fastRestart trusts the stamp, skips the storage walk, and never recovers.

Track per-repo outcomes via parseStats and stamp only when the walk fully populated the metaDB, otherwise log and continue so the next restart reparses

Signed-off-by: Jacob McSwain <jacob@mcswain.dev>

* fix(docs): readme: remove trailing comma from JSON config

Signed-off-by: Jacob McSwain <jacob@mcswain.dev>

* fix(meta): dynamodb: use context.Background instead of context.TODO

Signed-off-by: Jacob McSwain <jacob@mcswain.dev>

* fix(meta): invalidate fast restart on storage config changes

Signed-off-by: Jacob McSwain <jacob@mcswain.dev>

* chore(meta): dynamodb: use context.Background() instead of context.TODO()

Signed-off-by: Jacob McSwain <jacob@mcswain.dev>

* docs(meta): dynamodb: add comment about nil AttributeValue handling in GetWriterVersion

Signed-off-by: Jacob McSwain <jacob@mcswain.dev>

* chore: rename writer-version stamp to fast-restart stamp

also replaces the version/commit tracking to use BinaryVersion instead of WriterVersion

This should make things more clear

Signed-off-by: Jacob McSwain <jacob@mcswain.dev>

* fix(config): ensure FastRestart is on GlobalStorageConfig

This is not a per-subpath setting

Signed-off-by: Jacob McSwain <jacob@mcswain.dev>

* fix(metadb): redis: tests: ensure clients are closed

Signed-off-by: Jacob McSwain <jacob@mcswain.dev>

---------

Signed-off-by: Jacob McSwain <jacob@mcswain.dev>

pkg/test/mocks/repo_db_mock.go

@@ -46,6 +46,10 @@ type MetaDBMock struct {
 
 	PatchDBFn func() error
 
+	GetFastRestartStampFn func() (string, error)
+
+	SetFastRestartStampFn func(stamp string) error
+
 	ImageTrustStoreFn func() mTypes.ImageTrustStore
 
 	SetImageTrustStoreFn func(mTypes.ImageTrustStore)
@@ -171,6 +175,22 @@ func (sdm MetaDBMock) PatchDB() error {
 	return nil
 }
 
+func (sdm MetaDBMock) GetFastRestartStamp() (string, error) {
+	if sdm.GetFastRestartStampFn != nil {
+		return sdm.GetFastRestartStampFn()
+	}
+
+	return "", nil
+}
+
+func (sdm MetaDBMock) SetFastRestartStamp(stamp string) error {
+	if sdm.SetFastRestartStampFn != nil {
+		return sdm.SetFastRestartStampFn(stamp)
+	}
+
+	return nil
+}
+
 func (sdm MetaDBMock) GetStarredRepos(ctx context.Context) ([]string, error) {
 	if sdm.GetStarredReposFn != nil {
 		return sdm.GetStarredReposFn(ctx)