redis driver for blob cache information and metadb (#2865)

* feat: add redis cache support

https://github.com/project-zot/zot/pull/2005
Fixes https://github.com/project-zot/zot/issues/2004

* feat: add redis cache support

Currently, we have dynamoDB as the remote shared cache but ideal only
for the cloud use case.
For on-prem use case, add support for redis.

Signed-off-by: Ramkumar Chinchani <rchincha@cisco.com>

* feat(redis): added blackbox tests for redis

Signed-off-by: Petu Eusebiu <peusebiu@cisco.com>

* feat(redis): dummy implementation of MetaDB interface for redis cache

Signed-off-by: Alexei Dodon <adodon@cisco.com>

* feat: check validity of driver configuration on metadb instantiation

Signed-off-by: Andrei Aaron <aaaron@luxoft.com>

* feat: multiple fixes for redis cache driver implementation

- add missing method GetAllBlobs
- add redis cache tests, with and without mocking

Signed-off-by: Andrei Aaron <aaaron@luxoft.com>

* feat(redis): redis implementation for MetaDB

Signed-off-by: Andrei Aaron <aaaron@luxoft.com>

* feat(redis): use redsync to block concurrent write access to the redis DB

Signed-off-by: Andrei Aaron <aaaron@luxoft.com>

* feat(redis): update .github/workflows/cluster.yaml to also test redis

Signed-off-by: Andrei Aaron <aaaron@luxoft.com>

* feat(metadb): add keyPrefix parameter for redis and remove unneeded method meta.Crate()

Signed-off-by: Andrei Aaron <aaaron@luxoft.com>

* feat(redis): support RedisCluster configuration and add unit tests

Signed-off-by: Andrei Aaron <aaaron@luxoft.com>

* feat(redis): more tests for redis metadb implementation

Signed-off-by: Andrei Aaron <aaaron@luxoft.com>

* feat(redis): add more examples and update examples/README.md

Signed-off-by: Andrei Aaron <aaaron@luxoft.com>

* feat(redis): move option parsing and redis client initialization under pkg/api/config/redis

Signed-off-by: Andrei Aaron <aaaron@luxoft.com>

* chore(cachedb): move Cache interface to pkg/storage/types

Signed-off-by: Andrei Aaron <aaaron@luxoft.com>

* feat(redis): reorganize code in pkg/storage/cache.go

Signed-off-by: Andrei Aaron <aaaron@luxoft.com>

* feat(redis): call redis.SetLogger() with the zot logger as parameter

Signed-off-by: Andrei Aaron <aaaron@luxoft.com>

* feat(redis): rename pkg/meta/redisdb to pkg/meta/redis

Signed-off-by: Andrei Aaron <aaaron@luxoft.com>

---------

Signed-off-by: Ramkumar Chinchani <rchincha@cisco.com>
Signed-off-by: Petu Eusebiu <peusebiu@cisco.com>
Signed-off-by: Alexei Dodon <adodon@cisco.com>
Signed-off-by: Andrei Aaron <aaaron@luxoft.com>
Co-authored-by: a <a@tuxpa.in>
Co-authored-by: Ramkumar Chinchani <rchincha@cisco.com>
Co-authored-by: Petu Eusebiu <peusebiu@cisco.com>
Co-authored-by: Alexei Dodon <adodon@cisco.com>

pkg/cli/server/root.go

@@ -257,7 +257,9 @@ func validateCacheConfig(cfg *config.Config, log zlog.Logger) error {
 
 	if cfg.Storage.CacheDriver != nil && cfg.Storage.RemoteCache {
 		// local storage with remote database
-		if cfg.Storage.StorageDriver == nil {
+		// redis is supported with both local and S3 storage, while dynamodb is only supported with S3
+		// redis is only supported with local storage in a non-clustering scenario with a single zot instance,
+		if cfg.Storage.StorageDriver == nil && cfg.Storage.CacheDriver["name"] != storageConstants.RedisDriverName {
 			msg := "invalid database config, cannot have local storage driver with remote database!"
 			log.Error().Err(zerr.ErrBadConfig).Msg(msg)
 
@@ -265,7 +267,8 @@ func validateCacheConfig(cfg *config.Config, log zlog.Logger) error {
 		}
 
 		// unsupported database driver
-		if cfg.Storage.CacheDriver["name"] != storageConstants.DynamoDBDriverName {
+		if cfg.Storage.CacheDriver["name"] != storageConstants.DynamoDBDriverName &&
+			cfg.Storage.CacheDriver["name"] != storageConstants.RedisDriverName {
 			msg := "invalid database config, unsupported database driver"
 			log.Error().Err(zerr.ErrBadConfig).Interface("cacheDriver", cfg.Storage.CacheDriver["name"]).Msg(msg)