[BUG] only one pair of ak/sk is allowed in multi mounts Dataset when using JindoRuntime

[pandada8]

What feature you'd like to add:
Ability to specify different ak/sk for different mounts.
You can do this for now, but only the last aksk will work.
Why is this feature needed:
Mount different bucket from different account.

[TrafalgarZZZ]

Would you like provide more information about it? In Fluid, you can set spec.mounts[*].encryptOptions for each mount, so AFAIK it is a supported feature.

[pandada8]

The current implementation of jindocache actually prevents us from doing this.

fluid/pkg/ddc/jindocache/transform.go

Lines 470 to 494 in 6cc47b8

	if secretMountSupport {
	value.Secret = secretKeyRef.Name
	if key == "fs."+mountType+".accessKeyId" {
	value.SecretKey = secretKeyRef.Key
	e.Log.Info("Get %s From %s!", key, secretKeyRef.Name)
	}
	if key == "fs."+mountType+".accessKeySecret" {
	value.SecretValue = secretKeyRef.Key
	e.Log.Info("Get %s From %s!", key, secretKeyRef.Name)
	}
	} else {
	secret, err := kubeclient.GetSecret(e.Client, secretKeyRef.Name, e.namespace)
	if err != nil {
	e.Log.Info("can't get the input secret from dataset", secretKeyRef.Name)
	break
	}
	value := secret.Data[secretKeyRef.Key]
	if key == "fs."+mountType+".accessKeyId" {
	propertiesFileStore["jindocache."+mountType+".accessKeyId"] = string(value)
	}
	if key == "fs."+mountType+".accessKeySecret" {
	propertiesFileStore["jindocache."+mountType+".accessKeySecret"] = string(value)
	}
	e.Log.Info("Get Credential From Secret Successfully")
	}

Only single pair key/secret is set from the code

fluid/pkg/ddc/jindocache/transform.go

Lines 420 to 427 in 6cc47b8

	if os.Getenv("jindocache.internal.test") == "true" {
	if mount.Options["fs.oss.accessKeyId"] != "" {
	propertiesFileStore["jindocache.oss.bucket."+bucketName+".accessKeyId"] = mount.Options["fs.oss.accessKeyId"]
	}
	if mount.Options["fs.oss.accessKeySecret"] != "" {
	propertiesFileStore["jindocache.oss.bucket."+bucketName+".accessKeySecret"] = mount.Options["fs.oss.accessKeySecret"]
	}
	}

the debug code however support set different access key / secret key from options, but it's hidden under the jindocache.internal.test flag

[pandada8]

How to Reproduce:

apiVersion: data.fluid.io/v1alpha1
kind: JindoRuntime
metadata:
  name: merged
  namespace: default-group
spec:
  replicas: 1
  tieredstore:
    levels:
      - mediumtype: MEM
        path: /dev/shm
        volumeType: emptyDir
        quota: 20Gi
        high: "0.99"
        low: "0.95"
---
apiVersion: data.fluid.io/v1alpha1
kind: Dataset
metadata:
  name: merged
  namespace: default-group
spec:
  placement: "Shared"
  accessModes:
    - ReadWriteMany
  mounts:
    - mountPoint: oss://bucket-A/some-random-prefix/
      options:
        fs.oss.endpoint: oss-cn-beijing-internal.aliyuncs.com
      name: cache
      path: "/partA"
      readOnly: true
      encryptOptions:
        - name: fs.oss.accessKeyId
          valueFrom:
            secretKeyRef:
              name: oss-secret-A
              key: fs.oss.accessKeyId
        - name: fs.oss.accessKeySecret
          valueFrom:
            secretKeyRef:
              name: oss-secret-A
              key: fs.oss.accessKeySecret
    - mountPoint: oss://bucket-B/some-random-prefix/
      options:
        fs.oss.endpoint: oss-cn-beijing-internal.aliyuncs.com
      name: aegis
      path: "/partB"
      readOnly: true
      encryptOptions:
        - name: fs.oss.accessKeyId
          valueFrom:
            secretKeyRef:
              name: oss-secret-2
              key: fs.oss.accessKeyId
        - name: fs.oss.accessKeySecret
          valueFrom:
            secretKeyRef:
              name: oss-secret-2
              key: fs.oss.accessKeySecret
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: test-merged-mount
  namespace: default-group
spec:
  selector:
    matchLabels:
      app: test-merged-mount
  template:
    metadata:
      labels:
        app: test-merged-mount
    spec:
      containers:
      - name: test-merged-mount
        image: alpine
        command: ['sh', '-c', 'sleep infinity']
        resources:
          limits:
            memory: "128Mi"
            cpu: "500m"
        volumeMounts:
          - name: merged
            mountPath: /merged
      volumes:
        - persistentVolumeClaim:
            claimName: merged
          name: merged

Running in the pod

> ls /merged/
# hang
> ls /merged/partA/
# hang
> ls /merged/partB/
fileA fileB fileC

the generated jindo-fuse pod manifest contains only one reference to the secret, mounted at the /cache

...
    volumeMounts:
    - mountPath: /token/AccessKeyId
      name: jindofs-secret-token
      readOnly: true
      subPath: fs.oss.accessKeyId
    - mountPath: /token/AccessKeySecret
      name: jindofs-secret-token
      readOnly: true
      subPath: fs.oss.accessKeySecret
...
  volumes:
  - name: jindofs-secret-token
    secret:
      defaultMode: 420
      secretName: oss-secret-2

inside the jindo-fuse pod, only secret from the second mount can be found on the /token/

> ls /token/
AccessKeyId  AccessKeySecret

[TrafalgarZZZ]

@pandada8 Thanks for the detailed example for reproducing this issue. I think it's currently not supported in JindoRuntime. We will check it later how to support it.