Skip to content

Commit

Permalink
Support custom webhook format
Browse files Browse the repository at this point in the history
  • Loading branch information
@vtsykun
vtsykun committed Feb 25, 2020
1 parent 38f71b8 commit b4f2966
Show file tree
Hide file tree
Showing 2 changed files with 181 additions and 10 deletions.
154 changes: 154 additions & 0 deletions README.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -14,6 +14,7 @@ Features

- Compatible with composer.
- Support update webhook for GitHub, Bitbucket and GitLab.
- Support custom webhook format.
- Customers user and groups.
- Generic Packeton [webhooks](docs/webhook.md)
- Limit access by vendor and versions.
Expand Down Expand Up @@ -41,7 +42,15 @@ Table of content
- [JIRA issue fix version](/docs/webhook.md#jira-create-a-new-release-and-set-fix-version)
- [Gitlab setup auto webhook](/docs/webhook.md#gitlab-auto-webhook)
- [Ssh key access](#ssh-key-access-and-composer-oauth-token)
- [Update Webhooks](#update-webhooks)
- [Github](#github-webhooks)
- [GitLab](#gitlab-service)
- [GitLab Organization](#gitlab-group-hooks)
- [Bitbucket](#bitbucket-webhooks)
- [Manual hook](#manual-hook-setup)
- [Custom webhook format](#custom-webhook-format-transformer)
- [Usage](#usage-and-authentication)
- [Create admin user](#create-admin-user)

Demo
----
Expand Down Expand Up @@ -266,6 +275,140 @@ By default composer will use GitHub API to get metadata for your GitHub reposito
`use-github-api` to composer config.json to always use ssh key and clone the repository as
it would with any other git repository, [see here](https://getcomposer.org/doc/06-config.md#use-github-api)

Update Webhooks
---------------
You can use GitLab, GitHub, and Bitbucket project post-receive hook to keep your packages up to date
every time you push code.

#### Bitbucket Webhooks
To enable the Bitbucket web hook, go to your BitBucket repository,
open the settings and select "Webhooks" in the menu. Add a new hook. Y
ou have to enter the Packagist endpoint, containing both your username and API token.
Enter `https://<app>/api/bitbucket?token=user:token` as URL. Save your changes and you're done.

#### GitLab Service

To enable the GitLab service integration, go to your GitLab repository, open
the Settings > Integrations page from the menu.
Search for Packagist in the list of Project Services. Check the "Active" box,
enter your `packeton.org` username and API token. Save your changes and you're done.

#### GitLab Group Hooks

Group webhooks will apply to all projects in a group and allow to sync all projects.
To enable the Group GitLab webhook you must have the paid plan.
Go to your GitLab Group > Settings > Webhooks.
Enter `https://<app>/api/update-package?token=user:token` as URL.

#### GitHub Webhooks
To enable the GitHub webhook go to your GitHub repository. Click the "Settings" button, click "Webhooks".
Add a new hook. Enter `https://<app>/api/github?token=user:token` as URL.

#### Manual hook setup

If you do not use Bitbucket or GitHub there is a generic endpoint you can call manually
from a git post-receive hook or similar. You have to do a POST request to
`https://pkg.okvpn.org/api/update-package?token=user:api_token` with a request body looking like this:

```
{
"repository": {
"url": "PACKAGIST_PACKAGE_URL"
}
}
```

Also you can overwrite regex that was used to parse the repository url,
see [ApiController](src/Packagist/WebBundle/Controller/ApiController.php#L348)

```
{
"repository": {
"url": "PACKAGIST_PACKAGE_URL"
},
"packeton": {
"regex": "{^(?:ssh://git@|https?://|git://|git@)?(?P<host>[a-z0-9.-]+)(?::[0-9]+/|[:/])(scm/)?(?P<path>[\\w.-]+(?:/[\\w.-]+?)+)(?:\\.git|/)?$}i"
}
}
```

You can do this using curl for example:

```
curl -XPOST -H 'content-type:application/json' 'https://pkg.okvpn.org/api/update-package?token=user:api_token' -d' {"repository":{"url":"PACKAGIST_PACKAGE_URL"}}'
```

Instead of using repo url you can use directly composer package name.
You have to do a POST request with a request body.

```
{
"composer": {
"package_name": "okvpn/test"
}
}
```

```
{
"composer": {
"package_name": ["okvpn/test", "okvpn/pack2"]
}
}
```

#### Custom webhook format transformer

You can create a proxy middleware to transform JSON payload to the applicable inner format.
In first you need create a new Rest Endpoint to accept external request.

Go to `Settings > Webhooks` and click `Add webhook`. Fill the form:
- url - `https://<app>/api/update-package?token=user:token`
- More options > Name restriction - `#your-unique-name#` (must be a valid regex)
- Trigger > By HTTP requests to https://APP_URL/api/webhook-invoke/{name} - select checkbox
- Payload - Write a script using twig expression to transform external request to POST request from previous example.

For example, if the input request has a format, the twig payload may look like this:

```json
{
"repository":{
"slug":"vtsykun-packeton",
"id":11,
"name":"vtsykun-packeton",
"scmId":"git",
"state":"AVAILABLE",
"links": {
"clone": [
{"href": "https://github.com/vtsykun/packeton.git"}
]
}
}
}
```

```twig
{% set repository = request.repository.links.clone[0].href %}
{% if repository is null %}
{{ interrupt('Request does not contains repository link') }}
{% endif %}
{% set response = {
'repository': {'url': repository },
'packeton': {'regex': '{^(?:ssh://git@|https?://|git://|git@)?(?P<host>[a-z0-9.-]+)(?::[0-9]+/|[:/])(scm/)?(?P<path>[\\w.-]+(?:/[\\w.-]+?)+)(?:\\.git|/)?$}i'}
} %}
{{ response|json_encode }}
```

See [twig expression](docs/webhook.md) syntax for details.

Click the "Save button"

Now if you call the url `https://APP_URL/api/webhook-invoke/your-unique-name?token=user:token`
request will be forward to `https://APP_URL/api/update-package?token=user:token` with converted POST
payload according to your rules.

Usage and Authentication
------------------------
By default admin user have access to all repositories and able to submit packages, create users, view statistics.
Expand All @@ -279,6 +422,17 @@ composer config --global --auth http-basic.pkg.okvpn.org admin Ydmhi1C3XIP5fnRWc

API Token you can found in your Profile.

### Create admin user.

Only admin user can submit a new package and create the new customer users.
You can create admin user via console using fos user bundle commands.

```
php bin/console fos:user:create
php bin/console fos:user:promote <username> ROLE_ADMIN
```


LICENSE
------
MIT
37 changes: 27 additions & 10 deletions src/Packagist/WebBundle/Controller/ApiController.php
View file
Original file line number Diff line number Diff line change
Expand Up @@ -84,7 +84,7 @@ public function updatePackageAction(Request $request)
}

if (!$payload) {
return new JsonResponse(array('status' => 'error', 'message' => 'Missing payload parameter'), 406);
return new JsonResponse(['status' => 'error', 'message' => 'Missing payload parameter'], 406);
}

if (isset($payload['project']['git_http_url'])) { // gitlab event payload
Expand All @@ -99,9 +99,23 @@ public function updatePackageAction(Request $request)
$url = $payload['repository']['links']['html']['href'];
} elseif (isset($payload['canon_url']) && isset($payload['repository']['absolute_url'])) { // bitbucket post hook (deprecated)
$urlRegex = '{^(?:https?://|git://|git@)?(?P<host>bitbucket\.org)[/:](?P<path>[\w.-]+/[\w.-]+?)(\.git)?/?$}i';
$url = $payload['canon_url'].$payload['repository']['absolute_url'];
$url = $payload['canon_url'] . $payload['repository']['absolute_url'];
} elseif (isset($payload['composer']['package_name'])) { // custom webhook
$packages = [];
$packageNames = (array) $payload['composer']['package_name'];
$repo = $this->getDoctrine()->getRepository(Package::class);
foreach ($packageNames as $packageName) {
$packages = array_merge($packages, $repo->findBy(['name' => $packageName]));
}

return $this->schedulePostJobs($packages);
} else {
return new JsonResponse(array('status' => 'error', 'message' => 'Missing or invalid payload'), 406);
return new JsonResponse(['status' => 'error', 'message' => 'Missing or invalid payload'], 406);
}

// Use the custom regex
if (isset($payload['packeton']['regex'])) {
$urlRegex = $payload['packeton']['regex'];
}

return $this->receivePost($request, $url, $urlRegex);
Expand All @@ -114,6 +128,7 @@ public function updatePackageAction(Request $request)
* requirements={"package"="[A-Za-z0-9_.-]+/[A-Za-z0-9_.-]+?"},
* defaults={"_format" = "json"}
* )
* {@inheritDoc}
* @ParamConverter("package", options={"mapping": {"package": "name"}})
* @Method({"PUT"})
*/
Expand Down Expand Up @@ -293,15 +308,18 @@ protected function receivePost(Request $request, $url, $urlRegex)
return new Response(json_encode(['status' => 'error', 'message' => 'Could not parse payload repository URL']), 406);
}

// find the user
$user = $this->getUser();
if (!$user) {
return new Response(json_encode(['status' => 'error', 'message' => 'Invalid credentials']), 403);
}

// try to find the all package
$packages = $this->findPackagesByUrl($url, $urlRegex);

return $this->schedulePostJobs($packages);
}

/**
* @param Package[] $packages
* @return Response
*/
protected function schedulePostJobs(array $packages)
{
if (!$packages) {
return new Response(json_encode(['status' => 'error', 'message' => 'Could not find a package that matches this request (does user maintain the package?)']), 404);
}
Expand All @@ -310,7 +328,6 @@ protected function receivePost(Request $request, $url, $urlRegex)
$em = $this->get('doctrine.orm.entity_manager');
$jobs = [];

/** @var Package $package */
foreach ($packages as $package) {
$package->setAutoUpdated(true);
$em->flush($package);
Expand Down

0 comments on commit b4f2966

Please sign in to comment.