Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Backend for auto-updating APK #83

Open
wants to merge 1 commit into
base: master
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 3 additions & 0 deletions src/index.ts
Original file line number Diff line number Diff line change
@@ -1,6 +1,7 @@
export {};

import { getServersList } from './servers';
import { getLatestRelease } from './releases';

// Main entry point.
addEventListener('fetch', (event) => {
Expand All @@ -15,6 +16,8 @@ export async function handleRequest(request: Request) {
case '/resources': // Public for resources.
case '/servers':
return getServersList(request);
case '/releases':
return getLatestRelease(request);
}
return new Response('', { status: 404 });
}
95 changes: 95 additions & 0 deletions src/releases.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,95 @@
import { parseAppVersion, parseApkName } from './versions';

const GITHUB_RELEASES_URL: string = 'https://api.github.com/repos/organicmaps/organicmaps/releases';
// https://docs.github.com/en/rest/authentication/authenticating-to-the-rest-api?apiVersion=2022-11-28#authenticating-with-a-personal-access-token
const GITHUB_BEARER_TOKEN: string =
'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';
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Может лучше его из секретов подтягивать? Мало ли опенсорснем потом? Тогда лучше перегенерить.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Я посмотрю как убрать в секреты. Вообще пока кажется, что работает даже без него.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Тогда убрать, если не нужно. Релизы же публичные, ключ тоже тогда публичный может быть или вообще ненужен.


interface AppReleaseMetadata {
published_at: Date;
code: number;
flavor?: string;
type?: string;
apk: {
url: string;
biodranik marked this conversation as resolved.
Show resolved Hide resolved
name: string;
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Что будет в имени?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Название файла. Пока не уверен что прям остро необходимо, может не понадобится.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

filename тогда?

Что там ещё есть полезного?

size: number;
};
// TODO: figure out how to define map properly.
news: {
biodranik marked this conversation as resolved.
Show resolved Hide resolved
'en-US': string;
};
}

interface GitHubReleaseAssetMetadata {
browser_download_url: string;
name: string;
size: number;
content_type: string;
state: string;
}

interface GitHubReleaseMetadata {
published_at: Date;
draft: boolean;
prerelease: boolean;
body: string;
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Это описание воцнью на гитхабе? В каком формате? Ссылка на него есть?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Это текст из релиза на гитхабе в формате markdown текста.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Херово что маркдаун, у нас же нет рендерера на девайсе, верно?
Если простого решения нет, то придётся присылать ссылку на гитхаб страницу с этим текстом.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Рендерить маркдаун пока не будем, верно?

assets: [GitHubReleaseAssetMetadata];
}

export async function getLatestRelease(request: Request) {
const appVersion = parseAppVersion(request.headers.get('x-om-appversion'));
if (!appVersion) return new Response('Unknown app version', { status: 400 });

// The release version doesn't have `-release` suffix, thus type should be `undefined`.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Пояснишь, о каких версиях речь? У нас же много разных билдов и сборок, плюс не забывай, что все, кто нас форкает, будут тоже автоматом стучаться на мета сервер с их собственными версиями и названиями. Мы видели их уже в юзер агентах.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Релизные версии не имеют суффикса "-Release" в версии, поэтому type == undefined => "release".

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

А если в type мусор от всяких форков? Лучше явно проверять на наши типы.

if (appVersion.flavor != 'web' || appVersion.type !== undefined)
return new Response('Unknown app version', { status: 400 });

const response = await fetch(GITHUB_RELEASES_URL, {
cf: {
// Always cache this fetch (including 404 responses) regardless of content type
// for a max of 30 minutes before revalidating the resource
cacheTtl: 30 * 60,
cacheEverything: true,
},
headers: {
Accept: 'application/vnd.github+json',
'User-Agent': 'curl/8.4.0', // GitHub returns 403 without this.
'X-GitHub-Api-Version': '2022-11-28',
Authorization: `Bearer ${GITHUB_BEARER_TOKEN}`,
},
});
if (response.status != 200)
return new Response(`Bad response status ${response.status} ${response.statusText} ${response.body} from GitHub`, {
status: 500,
});

const releases = (await response.json()) as [GitHubReleaseMetadata];
const release = releases.find((release) => release.draft == false && release.prerelease == false);
if (release == undefined) return new Response('No published release in GitHub response', { status: 500 });

const apk = release.assets.find(
(asset) => asset.content_type == 'application/vnd.android.package-archive' && asset.name.endsWith('.apk'),
);
if (!apk) throw new Error('The latest release does not have APK asset');
const apkVersion = parseApkName(apk.name);
if (!apkVersion) throw new Error(`Failed to parse APK name: ${apk}`);
if (apkVersion.flavor != 'web' || apkVersion.type != 'release') throw new Error(`Unsupported APK name: ${apk}`);

const result: AppReleaseMetadata = {
published_at: release.published_at,
code: apkVersion.code,
news: {
'en-US': release.body,
},
apk: {
name: apk.name,
size: apk.size,
url: apk.browser_download_url,
},
};

return new Response(JSON.stringify(result), {
headers: { 'Content-Type': 'application/json' },
});
}
43 changes: 35 additions & 8 deletions src/versions.ts
Original file line number Diff line number Diff line change
Expand Up @@ -13,17 +13,40 @@ export function parseDataVersion(strDataVersion: string | null): number | null {
return dataVersion;
}

interface AppVersion {
code: number;
build?: number;
flavor?: string;
type?: string; // 'debug' | 'beta'
}

const APK_NAME_RE = /^OrganicMaps-(?<code>2\d{7})-(?<flavor>[A-Za-z3264]+)-(?<type>beta|debug|release)\.apk$/;

export function parseApkName(apkName: string): AppVersion | null {
const m = apkName.match(APK_NAME_RE);
if (m === null || !m.groups) return null;
const code = parseInt(m.groups.code);
if (Number.isNaN(code) || code < 20000000 || code > 30000000) return null;
const flavor = m.groups.flavor;
const type = m.groups.type;
const apkVersion: AppVersion = {
code: code,
flavor: flavor,
type: type,
};
return apkVersion;
}

// 2022.11.20 for iOS versions released before November 21 (without donate menu)
// 2022.11.24-4-ios for newer iOS versions (with donate menu)
// 2022.12.24-10-Google for Android
// 2022.12.24-10-Google-beta for Android
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is Web-beta supported?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

// 2022.12.24-3-3f4ca43-Linux or 2022.12.24-3-3f4ca43-dirty-Linux for Linux
// 2022.12.24-3-3f4ca43-Darwin for Mac
const VERSION_RE =
/(?<year>\d{4})\.(?<month>\d{1,2})\.(?<day>\d{1,2})(?:$|-(?<build>[0-9]+)(?:-[0-9a-f]+)?(?:-dirty)?-(?<flavor>[A-Za-z3264]+))/;
/(?<year>\d{4})\.(?<month>\d{1,2})\.(?<day>\d{1,2})(?:$|-(?<build>[0-9]+)(?:-[0-9a-f]+)?(?:-dirty)?-(?<flavor>[A-Za-z3264]+))(?:-(?<type>beta|debug))?/;
// Returns code like 221224 for both platforms, build and flavor for Android and newer iOS versions.
export function parseAppVersion(
versionName: string | null,
): { code: number; build?: number; flavor?: string | undefined } | null {
export function parseAppVersion(versionName: string | null): AppVersion | null {
if (!versionName) {
return null;
}
Expand Down Expand Up @@ -51,14 +74,18 @@ export function parseAppVersion(
return { code: code };
}

const buildNumber = parseInt(m.groups.build);
const build = Number.isNaN(buildNumber) ? 0 : buildNumber;
// 'ios' for iOS devices.
const flavor = (m.groups.flavor !== undefined && m.groups.flavor.toLowerCase()) || undefined;

return {
const appVersion: AppVersion = {
code: code,
flavor: flavor,
build: build,
};

const buildNumber = parseInt(m.groups.build);
if (!Number.isNaN(buildNumber)) appVersion.build = buildNumber;

if (m.groups.type !== undefined) appVersion.type = m.groups.type;

return appVersion;
}
57 changes: 57 additions & 0 deletions test/releases.test.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,57 @@
import { describe, expect, test } from '@jest/globals';
import { getLatestRelease } from '../src/releases';

describe('Get app release version for flavor', () => {
const flavors = ['2022.08.23-1-web'];
for (let flavor of flavors) {
test(flavor, async () => {
let req = new Request('http://127.0.0.1:8787/releases', {
headers: {
'X-OM-AppVersion': flavor.toLowerCase(),
},
});
const response = await getLatestRelease(req);
// TODO: How to print response.text in case of error?
expect(response.status).toBe(200);
const result = JSON.parse(await response.text());
expect(Number.parseInt(result.code)).toBeGreaterThanOrEqual(23040200);
expect(result.apk).toBeDefined();
});
}
});

describe('Unsupported flavors for app update checks', () => {
const unsupported = [
'garbage',
'',
'20220823',
'2022.08',
'2022.08.23', // Older iOS clients
'2022.08.23-1-Google-beta',
'2022.08.23-5-Google-debug',
'2022.08.23-1-fdroid-beta',
'2022.08.23-1-fdroid-debug',
'2022.08.23-1-web-beta',
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Этот надо поддерживать.

Или ввести новый флейвор, придумать хорошее название и сделать другой app id, чтобы ставился рядом с уже имеющимися fdroid и web и google и мог их заменить легко. Может даже импортнуть метки автоматом.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Текущий PR добавляет проверку релизов на GitHub. На GitHub выкладывается "web-release". Ничего другого туда не выкладывается.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Будет работать сейчас, после переименования?

'2022.08.23-1-web-debug',
'2022.08.23-1-Huawei-beta',
'2022.08.23-1-Huawei-debug',
// Mac OS version is not published yet anywhere.
'2023.04.28-9-592bca9a-dirty-Darwin',
'2023.04.28-9-592bca9a-Darwin',
];
for (let flavor of unsupported) {
test(flavor, async () => {
let req = new Request('http://127.0.0.1:8787/releases', {
headers: {
'X-OM-AppVersion': flavor.toLowerCase(),
},
});
try {
const response = await getLatestRelease(req);
expect(response.status).toBeGreaterThanOrEqual(400);
} catch (err) {
expect(err).toContain('Unsupported app version');
}
});
}
});
22 changes: 19 additions & 3 deletions test/versions.test.ts
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
import { describe, expect, test } from '@jest/globals';
import { parseDataVersion, parseAppVersion } from '../src/versions';
import { parseDataVersion, parseAppVersion, parseApkName } from '../src/versions';

describe('parseDataVersion', () => {
const tests: { [key: string]: number | null } = {
Expand All @@ -18,6 +18,22 @@ describe('parseDataVersion', () => {
test('', () => expect(parseDataVersion(null)).toEqual(null));
});

describe('parseApkName', () => {
const tests: { [key: string]: object | null } = {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

В моём PR было больше тестов на не поддерживаемые кейзы https://github.com/organicmaps/meta/pull/65/files#diff-4a1ee83ef3a98a79eeaeb2506070a86479a12bad67c28ba34045f71159794373R35

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Разве там был парсинг имени APK?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Там тесты на кривые x-om-appversion.

'OrganicMaps-24020611-web-release.apk': { code: 24020611, flavor: 'web', type: 'release' },
'OrganicMaps-24020611-web-release': null,
'OrganicMaps-24020611-web-.apk': null,
'OrganicMaps-24020611- -.apk': null,
'OrganicMaps-2402061-web-release.apk': null,
garbage: null,
'': null,
null: null,
};
for (const input in tests) {
test(input, () => expect(parseApkName(input)).toEqual(tests[input]));
}
});

describe('parseAppVersion', () => {
const tests: { [key: string]: object | null } = {
// Older iOS releases without donate menu
Expand All @@ -27,8 +43,8 @@ describe('parseAppVersion', () => {
// There were no such versions in production.
'2022.08.01-1': null,
'2022.08.01-1-Google': { code: 220801, build: 1, flavor: 'google' },
// -debug is ignored
'2022.08.01-1-Google-debug': { code: 220801, build: 1, flavor: 'google' },
'2022.08.01-1-Google-debug': { code: 220801, build: 1, flavor: 'google', type: 'debug' },
'2022.08.01-1-Google-beta': { code: 220801, build: 1, flavor: 'google', type: 'beta' },
// TODO: Fix regexp. Not it should not happen in production.
//'2022.08.01-1-fd-debug': { code: 220801, build: 1, flavor: 'fd' },
'2022.1.1-0': null,
Expand Down
Loading