Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add exists method to storage #266

Open
caseycrogers opened this issue Aug 9, 2022 · 18 comments
Open

Add exists method to storage #266

caseycrogers opened this issue Aug 9, 2022 · 18 comments
Labels
enhancement New feature or request

Comments

@caseycrogers
Copy link

caseycrogers commented Aug 9, 2022

I would like to be able to check if a file exists in Supabase storage without having to download it:
final bool fileExists = await supabase.storage.from('some-bucket-id').exists('/some/path');

@caseycrogers caseycrogers changed the title Add "exists" method to storage Add exists method to storage Aug 9, 2022
@Vinzent03
Copy link

There is currently no api enpoint for this. See the documentation. For the time being, you can create your own postgres function, which queries the storage.objects table.

@dshukertjr
Copy link
Member

dshukertjr commented Aug 11, 2022

@caseycrogers
Thanks for the feedback!

As @Vinzent03 has said, there is currently no APIs for that, so I would recommend creating an issue on the Storage API repo.

@dshukertjr
Copy link
Member

Come to think about it, what is the use case for this method? In what scenario would you not know if a file exists or not?

The way use storage is that after I upload a file, I either save the public URL or the path to the file in a database, and when I need to access the file, I would just access it via the saved URL or generate a URL from the saved path.

@Vinzent03
Copy link

You may have paths, where you don't have to save the path, because it's not random, but constructed from other parameter. For example, you may have a directory with profile images, where the file name is the uid of the user. You don't have to specifically save the filename, because you already know the uid.

@dshukertjr
Copy link
Member

@Vinzent03
In that example, if they had saved the path on the database, they would save 1 extra http request, hence would be able to speed up whatever process that follows.

I fear that adding a exists method might steer the developers toward bad practices, where they would start making extra requests that can be mitigated, but what do you think?

@Vinzent03
Copy link

@dshukertjr I'm not sure if you fully understood my use case. Let's say profile images are stored in a predefined directory with the user's uid as the file's name. How do you know on the client whether you already uploaded an image or not? I think the only ways are providing an extra method in the client library, or the user uses a head request and checks for 404.

@dshukertjr
Copy link
Member

dshukertjr commented Dec 8, 2022

@Vinzent03
I think I do understand the use case now. I was referring to a use case where you want to display other users' profile images, but I believe you are referring to a use case where you are displaying the logged in user's own profile image.

At least for now, there are some workarounds like doing a head request or to use a erorrBuilder on the profile image to show a fallback image in case there are no profile image set yet!

Let's keep this issue open to see if more users demand this feature!

@jherman
Copy link

jherman commented Dec 14, 2022

Ran into this same exact use case today, so just want to give my +1 to this feature getting implemented.

While I do have a fallback image, I'd rather not show the request error in the console if it could be avoided.

EDIT:

Just wanted to include the database postgres function call I created for those stuck on this:

BEGIN RETURN
  (SELECT EXISTS
     (SELECT id
      FROM storage.objects
      WHERE bucket_id='avatars'
        AND OWNER=UID
        AND name=PATH
      LIMIT 1));

END;

And the call to it:

  const { data, error } = await supabase.rpc('storage_avatar_exists', {
    uid: '542eb4f2-c890-47d0-a32c-4ebba849a011',
    path: '542eb4f2-c890-47d0-a32c-4ebba849a011/avatar.png'
  });

@dshukertjr dshukertjr added the enhancement New feature or request label Jan 20, 2023
@dshukertjr dshukertjr transferred this issue from supabase/supabase-flutter Jan 20, 2023
@dshukertjr
Copy link
Member

Transferring this issue to the storage-api repo to discuss its implementation on the backend.

@fenos
Copy link
Contributor

fenos commented Jan 23, 2023

We already have a backend method to check the existence of an object, simply using a HEAD request instead of a GET on a specific object will return empty content if the item exists else 404

We should probably update the client SDK to implement this method

@shriharip
Copy link

Running into this issue. User Avatars.
Bucket is a public bucket, so I do not generate the url to save it to db.
I generate the url dynamically,
baseStorageurl + sessionid + avatar.jpg

I want to check if the folder exists before i create the img source url like above.

@revulvgina
Copy link

We already have a backend method to check the existence of an object, simply using a HEAD request instead of a GET on a specific object will return empty content if the item exists else 404

We should probably update the client SDK to implement this method

tried now using HEAD on a deleted file, still 200.
tried GET, got 400, with status 404 on json message
image

@softmarshmallow
Copy link

And why is this yet not available?

@phoenixbox
Copy link

phoenixbox commented Mar 5, 2024

Authed HEAD request for reference

const response = await fetch(`${process.env.SUPABASE_URL}/storage/v1/object/${bucket}/${path}`, {
  method: 'HEAD',
  headers: {
    authorization: session.access_token,
  },
});

@utkarsh1097
Copy link

This will be a useful method. It looks like Google Cloud Storage (and by extension, Firebase Storage?) has a similar method.

@Owez
Copy link

Owez commented Jun 22, 2024

This is 100% needed, basic feature of storage that doesn't exist yet

@softmarshmallow
Copy link

softmarshmallow commented Jun 26, 2024

Not a good way yo handle things, but for those who does not care about performance that much and looking for batch types of work, heres some extended utilities.

import type { SupabaseClient } from "@supabase/supabase-js";
import type { FileObject } from "@supabase/storage-js";

export namespace SupabaseStorageExtensions {
  type StorageClient = SupabaseClient["storage"];

  async function list(storage: StorageClient, bucket: string, path: string) {
    const { data, error } = await storage.from(bucket).list(path);

    if (error) {
      throw error;
    }

    return data;
  }

  export async function tree(
    storage: StorageClient,
    bucket: string,
    path = ""
  ) {
    let files: Record<string, FileObject> = {};
    let stack: string[] = [path];

    while (stack.length > 0) {
      const currentPath = stack.pop();
      const items = await list(storage, bucket, currentPath!);

      const promises = items.map(async (item) => {
        const itemPath = currentPath + "/" + item.name;
        const isfile = !!item.id;
        if (isfile) {
          files[itemPath] = item;
        } else {
          stack.push(itemPath);
        }
      });

      await Promise.all(promises);
    }

    return files;
  }

  export async function rmdir(
    storage: StorageClient,
    bucket: string,
    path: string
  ) {
    const files = await tree(storage, bucket, path);
    const paths = Object.keys(files);
    const { data, error } = await storage.from(bucket).remove(paths);

    if (error) {
      throw error;
    }

    return data;
  }

  export async function exists(
    storage: StorageClient,
    bucket: string,
    path: string
  ) {
    const { data, error } = await storage.from(bucket).list(path);

    if (error) {
      throw error;
    }

    return data.some((file) => file.name === path.split("/").pop());
  }
}

Alternatively, you can use createSignedUrl, it will return error when used on non-exist object.

@PaperPrototype
Copy link

PaperPrototype commented Aug 12, 2024

@dshukertjr

I demand this feature 😛 I am intentionally avoiding saving a path_to_file and instead I have a uid for profiles, that I then construct whenever I need it.

getPublicUrl may only be constructing a URL and not actually querying, but maybe querying internal storage schema and store something like result.data.exists?

if (profiles) {
	for (let i = 0; i < profiles.length; i++) {
		const result = supabase.storage
			.from('profiles')
			.getPublicUrl(`/profiles/${profiles[i].id}/preview.png`, {
				transform: t200by200
			});
		
		// getPublicUrl may only be constructing a URL and not actually querying, 
		// but maybe query internal storage schema and store something like `result.data.exists`
		console.log(result.data.exists);
		if (result.data.exists) {
		         profiles[i]['image_url'] = result.data.publicUrl;
		}
	}

	profiles = data;
}

Although I can definitely see how this feature could be abused and result in way too many API calls just to check if they should put a URL for an image into a component. It's way cheaper to just leave an img_url even if it fails than query each image and check if it is available.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

No branches or pull requests