List of storage objects

[Loredani68]

I miss the possibility to retrieve a list of files in a specific storage folder.
An addition to the storage library for this would be fabulous.

The corresponding API function exists (https://cloud.google.com/storage/docs/json_api/v1/objects/list?hl=en).

[SchneiderInfosystems]

I will check this point.
Unfortunately, this interface is not explained in the Firebase Storage REST API description. That is why I did not found.

[Loredani68]

Yes, I understand that. With all these platform-specific references, it's really not easy to keep track.
In any case, I would be very happy if this feature could be added.

[SchneiderInfosystems]

This feature is not solvable: Unfortunately, this REST API is not available for using the Firebase token ID. When I try the Rest API with Firebase token retrieved from the Firebase Auth, I get "unauthenticated error" 401.

This REST API would require a Google Auth 2.0 Token (That is different from the Firebase token ID). To get such a token, we would need admin credentials from a service account to your Firebase project. You should never deploy this admin credentials within a client application. Of course, you could build this way a service application for your internal use. This application is out of scope for FB4D because it is too dangerous that somebody delivers the admin credentials within a client application to their customers.

Unfortunately, my explanations in the introduction of the wiki to the Storage (https://github.com/SchneiderInfosystems/FB4D/wiki/Getting-Started-with-FB4D#first-steps-with-the-storage) are still valid. I have written now a change request to the Firebase support but I'm skeptical. I'm uncomfortable to have no better solution.

[Loredani68]

All right. Thank you very much for the detailed explanations.
Google is probably not quite finished with the integration of the purchased Firestore into the native Google environment (if this is the plan at all...).

[macc2010]

https://cloud.google.com/storage/docs/json_api/v1/objects/list

[SchneiderInfosystems]

I will check this approach. Thank you

[SchneiderInfosystems]

My problem is that this REST API does not work for Firebase user tokens. See my question here: https://stackoverflow.com/questions/73584914/how-do-i-get-access-to-google-cloud-storage-services-with-a-firebase-user
@Max2010: Can check my current approach within your Firebase project?

function TFirebaseStorage.ListSynchronous(
  const ObjectPath: string): string; 
var
  Request: IFirebaseRequest;
  Response: IFirebaseResponse;
begin
  Request := TFirebaseRequest.Create(BaseURL, '', fAuth);
  Response := Request.SendRequestSynchronous([ObjectPath], rmGet , nil, nil, tmBearer); 
  if not Response.StatusOk then
  begin
    {$IFDEF DEBUG}
    TFirebaseHelpers.Log('FirebaseStorage.ListSynchronous: ' +
      Response.ContentAsString);
    {$ENDIF}
    raise EStorageObject.CreateFmt('List failed: %s', [Response.StatusText]);
  end else begin
    Response.CheckForJSONObj;
    result := Response.ContentAsString;
  end;
end;
 

This approach only works for me if I set the permission on the storage bucket for allUser in the Cloud Console (not in the Firebase Console). But this makes the bucket publicly readable. I don't think this is a solution.

[macc2010]

For me, this works :

function TFirebaseStorage.GetListSynchronous( const aFolder : string = '' ) : TStringList;
var
  QueryParams: TQueryParams;
  Request: IFirebaseRequest;
  Response: IFirebaseResponse;
  ResObj: TJSONObject;
  ResArr: TJSONArray;
  I : Integer;
  ItemVal : TJSONValue;
  sFolder, sName : string;
begin
  // https://cloud.google.com/storage/docs/json_api/v1/buckets/list
  QueryParams := TQueryParams.Create;
  try
    Request := TFirebaseRequest.Create( Format( 'https://firebasestorage.googleapis.com/v0/b/%s/o', [ fBucket ] ), '', fAuth );
    // if aFolder is empty ( the default value ), this function get all files of the bucket
    if aFolder <> '' then
    begin
      if aFolder = '/' then
        // list all files and folders of the root path
        QueryParams.Add( 'delimiter', [ '/' ] )
      else
      begin
        // list all files of a folder
        sFolder := aFolder;
        if Copy( sFolder, 1, 1 ) = '/' then
          sFolder := Copy( sFolder, 2, Length( sFolder ) - 1 );
        if Copy( sFolder, Length( sFolder ), 1 ) = '/' then
          sFolder := Copy( sFolder, 1, Length( sFolder ) - 1 );
        QueryParams.Add( 'delimiter', [ '/' ] );
        QueryParams.Add( 'prefix', [ sFolder + '/' ] );
      end;
    end;
    Response := Request.SendRequestSynchronous([], rmGet, nil, QueryParams);
  finally
    QueryParams.Free;
  end;
  Response.CheckForJSONObj;
  ResObj := Response.GetContentAsJSONObj;
  try
    // ResObj example :
    // {
    //   "kind": "storage#objects",
    //   "nextPageToken": "Cj1hZGVsaW5vdmlsbGFycm9tZXJvQGdtYWlsLmNvbS9mYWN0dXJhLWhjY2FkaXpfbWFnbmEtVk8tMjAucGRm", <---- this is responsed if we send maxResults
    //   "items": [
    //     {
    //       "kind": "storage#object",
    //       "id": "projectid.appspot.com/VO-318.pdf/1661841008266071",
    //       "selfLink": "https://www.googleapis.com/storage/v1/b/projectid.appspot.com/o/VO-318.pdf",
    //       "mediaLink": "https://content-storage.googleapis.com/download/storage/v1/b/projectid.appspot.com/o/VO-318.pdf?generation=1661841008266071&alt=media",
    //       "name": "VO-318.pdf",
    //       "bucket": "projectid.appspot.com",
    //       "generation": "1661841008266071",
    //       "metageneration": "2",
    //       "contentType": "application/pdf",
    //       "storageClass": "STANDARD",
    //       "size": "11843",
    //       "md5Hash": "wTl/1S+aB5bPEpKlWaJpiQ==",
    //       "crc32c": "ecGAhA==",
    //       "etag": "CNeW4pf47fkCEAI=",
    //       "timeCreated": "2022-08-30T06:30:08.289Z",
    //       "updated": "2022-08-30T06:30:27.359Z",
    //       "timeStorageClassUpdated": "2022-08-30T06:30:08.289Z",
    //       "metadata": {
    //         "firebaseStorageDownloadTokens": "642c1d08-edba-44be-89e3-660f100db129"
    //       }
    //     },
    //     {
    //       ....
    //       "name": "VO-202.pdf",
    //       ....
    //     }
    //   ]
    // }
    Result := TStringList.Create;
    try
      if ResObj.TryGetValue( 'items', ResArr ) then
      begin
        for I := 0 to ResArr.Count - 1 do
        begin
          ItemVal := ResArr.Items[ I ];
          if ItemVal.TryGetValue( 'name', sName ) then
            Result.Add( sName );
        end;
      end;
      // "prefixes": [ <----- responsed when aFolder = '/'
      //     "folder1/",
      //     "folder2/",
      //     "folder3/"
      // ]
      if ResObj.TryGetValue( 'prefixes', ResArr ) then
      begin
        for I := 0 to ResArr.Count - 1 do
        begin
          ItemVal := ResArr.Items[ I ];
          Result.Add( ItemVal.Value );
        end;
      end;
    except
      FreeAndNil( Result );
      raise;
    end;
  finally
    ResObj.Free;
  end;
end;

[macc2010]

My problem is that this REST API does not work for Firebase user tokens. See my question here: https://stackoverflow.com/questions/73584914/how-do-i-get-access-to-google-cloud-storage-services-with-a-firebase-user @Max2010: Can check my current approach within your Firebase project?

function TFirebaseStorage.ListSynchronous(
  const ObjectPath: string): string; 
var
  Request: IFirebaseRequest;
  Response: IFirebaseResponse;
begin
  Request := TFirebaseRequest.Create(BaseURL, '', fAuth);
  Response := Request.SendRequestSynchronous([ObjectPath], rmGet , nil, nil, tmBearer); 
  if not Response.StatusOk then
  begin
    {$IFDEF DEBUG}
    TFirebaseHelpers.Log('FirebaseStorage.ListSynchronous: ' +
      Response.ContentAsString);
    {$ENDIF}
    raise EStorageObject.CreateFmt('List failed: %s', [Response.StatusText]);
  end else begin
    Response.CheckForJSONObj;
    result := Response.ContentAsString;
  end;
end;
 

This approach only works for me if I set the permission on the storage bucket for allUser in the Cloud Console (not in the Firebase Console). But this makes the bucket publicly readable. I don't think this is a solution.

You can use custom claims to write a rule that only super admin users can list the storage, or you can ask in the rules if the user is [email protected] for example and allow in this case to list the bucket.

Also, the allow list rule is there : https://firebase.google.com/docs/storage/security/core-syntax.

[SchneiderInfosystems]

Have you manually set the permission on the storage bucket for allUser? Then my understanding is that the Storage Rules are bypassed. Can you check this point?

How you can check custom claims in the rules?

[macc2010]

In my case, I have written a rule to allow to an email super user ( [email protected] ) to be able to list files.

Example :
allow list: if (request.auth.uid != null) && (request.auth.token.email == "[email protected]";

But in a future, I want to set custom claims for this super admin users ( via Admin SDK ), for example, if I set a custom claim like : superadmin = true, in the storage rules I can use it :

allow list: if request.auth.token.superadmin == true;

[SchneiderInfosystems]

I got some answers from the google cloud community staff to my question relating Cloud Storage API usage with Firebase Auth:

https://www.googlecloudcommunity.com/gc/Security/How-do-I-get-access-to-Google-Cloud-Storage-services-with-a/td-p/462542
https://firebase.google.com/docs/storage/web/list-files

I would like to try the Javascript solution and analyse the network communication.

Unfortunately, I’m currently completely overloaded with the work on Vision ML Interface showing at the next EKON in Düsseldorf, Germany.