[Case Study] Bintray Automation for a Tedious Situation: Automating Package Distribution

Did you ever find yourself spending large amounts of time on a manual process, just wishing there was an automatic way to do it?

This case study will take you through how James Ward, Engineering and Open Source Ambassador at Salesforce.com, developed an automated process using JFrog Bintray for on-demand deployments of WebJars, that are created, distributed and available within the community.

After realizing that his many hours of manually deploying WebJars during his free time were unrealistic to maintain in the long run, Ward chose to use Bintray’s extensive and easy-to-use REST API to automate the process and make it available to the community on demand using JCenter, Bintray’s public Maven repository.

Ward’s WebJars project is a classic example of how JFrog Bintray can be used to scale up a tedious and time-consuming manual procedure into an easy and automated one.

Download the case study to learn more on how James Ward chose Bintray to scale thousands of WebJar Files.

The ABCs of Distributing Android Libraries

ABCDistributionFeatureBintray’s central repositories, JCenter and Conan-Center, are binary hubs for public OSS Maven and Conan (C++) packages respectively. They offer a great channel to distribute your public OSS packages. Having been around for a while, JCenter has become one of the most comprehensive sources for public OSS Maven packages, and is the channel-of-choice for many Maven projects. So we weren’t surprised to discover this blog post that describes how easy it is to distribute Android libraries through Bintray and JCenter.

Distributing Android libraries (or any other Maven package for that matter) through JCenter has its advantages. As one of the most popular Maven hubs, it exposes your package to a huge audience, and yet, you retain full ownership and can control how your package evolves. Basically, you upload your package to one of your public Maven repositories and ask for it to be included in JCenter. Once approved by the Bintray team, your package will be searchable on JCenter and freely available for download. You can learn about the details in the blog or look it up in the Bintray User Manual.

So if you’re ready to give your package world-wide fame, go ahead and open an account on Bintray. Bear in mind that since JCenter is a curated repository with trusted content, you need to be on a fully registered OSS or Premium account (as opposed to being on a trial) before you can link your package.

To start with a free OSS account, you can sign up here.

If you’re ready to go Premium (Pro or Enterprise), go here.

And if you want the full Enterprise experience, without committing to it yet, you can start a free trial (just remember to switch to an OSS or Premium account if you want to use JCenter).

Time is of the essence: Make an impact using Firehose Events

FirehoseFeatureHave you ever experienced the drive you get when you act on a sudden opportunity that really makes a valuable impact on something? On the other hand, have you ever experienced the disappointment of a missed opportunity?

Let’s face it, every user action can be translated into a short window of opportunity that requires our attention, otherwise, it will be lost forever. Every department in an organization is responsible for ensuring the best possible user experience, and many times we only get one chance to create an impactful initial impression that really makes the difference. So, how can we do this? One of the ways to do this is using JFrog Bintray’s Firehose Events.

We are constantly tracking, measuring and analyzing our user’s behaviors. Most of the time we look back at the user activity that has already happened and try to figure out where we can make improvements. For example, we can measure the number of downloads for any given time, and get valuable insights into the usage of our repositories using JFrog Bintray’s Premium Dashboard. We can even use Live Logs to get detailed stats, including a live download feed as it happens. But what about looking at what’s happening right now, in real-time

What are firehose events?

Bintray’s Firehose Events API enables us to receive live notifications (event triggers / user actions) for a variety of interactions with repositories, and respond to them in real-time with automated activities. Here are the events that you can register for, and some interesting things you can do with them:

  • Successful and failed logins: Repeated failed login attempts for scoped users can trigger an email alert to administrators
  • Downloading a file/artifact: Successful downloads can trigger an engagement email
  • Uploading a file/artifact: Very large uploads can trigger an email alert
  • Usage thresholds: Approaching the storage limit for your repositories can trigger an instant message
  • Deleting a file/artifact: Deleted packages can trigger an alert

Wrapping up with JFrog CLI

Yet again, JFrog CLI provides a convenient and simple interface for your automation scripts. It wraps the Firehose Events API and connects to an Event Notification Stream offering two main advantages in doing so:

  • Automatic reconnect: If the connection to Bintray is lost, JFrog CLI will automatically reconnect you to make sure that none of the events are lost.
  • Event filtering: You might not be interested in all the events coming out of the firehose. The CLI lets you filter out events, by event type, letting you focus only on those that are interesting to you.

Let’s take a look at a simple example. We can use the Firehose Events API to post to a Slack channel and provide a live stream of notifications.

To get events posted to a slack channel, all you have to do is first create a Slack channel, then connect to the event notification stream (Firehose), and pipe the stream to the Slack channel using something as simple as cURL calling the Slack API.

This example uses Bash, JFrog CLI, jq, and cURL. First, you’ll need to install JFrog CLI. Then wrap the following snippet within a script, and run it:

# Connect to the event notification stream
./jfrog bt st  --user= --key= --include="download" |
while read line
# Extract the path from the whole event response
  path=$(echo $line | jq .path -r)
  curl   -H"Content-Type:application/json" --data "{\"text\":\"File Downloaded: $path\"}"

This is just a simple example of how you can react to Firehose API events. The more creative and inventive Bintray users may come up with ideas like running analytics, creating live graphs and dashboards, issuing relevant alerts, and anything else your imagination can conjure up.

Upgrade to Bintray Enterprise!

What Makes a Dashboard “Premium”?

280x215.pngOnce you’ve uploaded a package to Bintray, the one thing you want to see, more than anything else, is downloads. Without downloads, your package is like the proverbial tree that falls down in an empty forest, the proverbial sound of one hand clapping, the … well, you get the picture. Bintray is happy to give you download stats, color-coded per version and all, even for packages in an OSS repository.


While a chart like the one above is a great way to start your morning, eventually you find yourself wanting more, especially if you’re an enterprise with many repositories, even more packages, and (hopefully) lots of customers (both internal and the paying kind).

This is where Bintray’s Premium Dashboard steps up to the plate.

Once you go “Premium”, Bintray shows you three different views of usage:

Total Usage

This is identical to the download statistics you got before going Premium.

By Repository

Here’s where things start to get more interesting. You get a breakdown of downloads and storage for each repository in your account. This adds an important dimension to your total usage. If you see peaks of usage (which may add up in your monthly bill), you can now identify exactly which repositories are generating those downloads. This lets you make an informed decision on how to handle those repositories. Perhaps you should change your Bintray plan to include premium repositories or Geo and IP restrictions, or maybe you should just move certain packages to a different repository and distribute usage better. It’s like the difference between just getting the total charge on your phone bill, as opposed to getting an itemized list of calls you made. Once you have the information, you can make informed decisions on what to do.


By Business Unit

And here’s where things get really interesting. How can you monitor downloads of your software and storage used per customer; whether it’s a paying customer who pays per download volume or an internal “customer” that you are monitoring for usage? Well, you could assign a separate repository for each customer. You might even think up some neat naming convention for repositories that shows to which customer each repository was assigned. But what if each customer downloads from a number of repositories? Then, you could keep a note of all the repositories assigned to a customer and do the math. That might work, but it might also get out of control when your organization gets big enough with many repositories serving many customers (the problems we all want to have). There’s no need to develop elaborate schemes to handle your billing; Bintray does all the math for you with Business Units. If you define each customer as a Business Unit, you can then assign all the repositories that serve that customer to the corresponding Business Unit, and get usage stats per Business Unit (i.e. per customer).


But wait; there’s more

What if one of your customers is downloading too much? “Why is this a problem?” you ask. Well, not all customers are the paying kind. For example, you may have a business unit assigned to a contractor who is doing some work for you, and you need to monitor and limit usage of the repositories you assigned to your contractor. If you’re on an Enterprise plan, Bintray lets you set usage thresholds using the REST API. Through the Usage Thresholds REST API endpoints you can set a threshold on maximum monthly storage, monthly download volume or daily download volume, and get alerted when any of these thresholds is exceeded. The alerts are delivered as firehose events, but you can also specify any number of email addresses that should also receive the alert.

So there’s your answer. A dashboard provides you with information. A premium dashboard gives you different ways to analyze that information giving you valuable insights into usage of your repositories. And Bintray’s premium dashboard also gives you control over how your repositories are used with usage thresholds. You see, there’s “premium”, and then there’s “Bintray Premium”.

Have an account? Go Premium

Don’t have an account? Sign up for a free trial

Securely Onboarding Colleagues through SAML Authentication

bintray_saml280x215.pngOnce you’ve created your Bintray account, getting your colleagues on board with permission-based access to your organization’s content is not always so easy. You want to use the most secure authentication available, so why can’t you use your corporate SAML server to authenticate your users?

The answer is, now you can.

If you configure your Bintray organization with the details of your SAML server, your colleagues can simply log in using their corporate SSO credentials, and they’re automatically included in your organization. So, not only are you using the most secure authentication your organization has to offer, you’ve also made it easy for your colleagues to get on board.interacting_saml_config_blurred

Now, you can make sure each user is assigned the right permissions by adding them to the corresponding Teams through the UI. That takes care of your colleagues and teammates, but what about external contractors or customers to whom you want to give access? They can’t be managed with teams and permissions since they’re not part of your Bintray organization  (and may not have a Bintray account at all). The answer is to use entitlements defined through the REST API giving them access according to a specified scope – these are what we call “scoped users”. More about that in a future post coming soon.

IP Restriction with White CIDR and Black CIDR

IP restrictionImagine this scenario. Your flagship product is doing OK; you’re getting downloads. Nevertheless, increasing sales is always a top priority, so you decide to create a free OSS version to boost usage and generate more awareness in the market. It’s also a great product, free to download, and is a great teaser for the upsell to your commercial version. So you upload the OSS version to your organization’s public repository on Bintray. After a while, you check its stats and are delighted to see thousands of downloads, and yet, there’s no buzz. If so many people are downloading your package, why is nobody talking about it? After checking your logs, you discover that most of those downloads you were so happy about are from the same set of IP addresses.

Someone is reverse-spamming you, and you need to block them.

Geo-Restriction could do the job, but then you don’t want to restrict whole countries at a time. You need something more targeted. Bintray’s new capability for IP restriction will do the job.

IP restriction is very similar to geo-restriction, only instead of defining your whitelist and blacklist as countries, you define them as a range of IP addresses using CIDRs. This gives you much more control letting you allow or block download at any level of granularity down to a single specific IP address. Whatever IP those bots are operating from can now be easily blocked so you can get the real download statistics you were interested in.

You can define your CIDR whitelist and blacklist through Bintray’s REST API or in the Edit Repository page.

IP restriction

Note that if there are any overlapping IPs, (i.e. IP addresses that are included in CIDRs that are in both the whitelist and the blacklist), then blacklisting them takes precedence. And if you accidentally list the exact same CIDR in both the whitelist AND the blacklist, Bintray will catch that and issue an error.

Ready to use IP restrictions? Get a quote for an enterprise account and ask for a free trial.

Automated EULA-protected Downloads

BINTRAY EULA280x215One of the great features of a Bintray Enterprise account is the ability to present a EULA when a user downloads one of your Products. That works well for the general case when the downloading user is a real person who can go through the Bintray UI and physically accept the EULA for publicly available content. This is something we discussed in a previous post. But what if you want to be more selective about who can download your content, and you want to limit it to a defined set of people? And what if those people want to download your content using automated scripts? And what if they don’t even have an account on Bintray? Those are a lot of hoops to jump through, but Bintray now offers a solution to this common circus of circumstances.

You can now offer automated EULA-protected downloads through the REST API to any pre-defined set of users, whether they have a Bintray account or not.

Here’s how it works.

  • You create an entitlement which gives access to your product and then you provide the corresponding access key to your user.
  • Your user browses to a specially constructed Bintray URL
  • Bintray displays a login screen where your user enters the access key as the username, and its password
  • Bintray then displays the EULA for your user to accept.


From now on, any REST API call that includes the access key you provided to your user will succeed unhindered by the EULA, since the EULA has already been accepted using that key. In other words – automated EULA-protected download. And who gets this privileged access? Not everyone. Only those who are privileged enough to receive the access key you created to protect your content. In other words private, automated EULA-protected download.

Protecting your products with a EULA has never been easier, whether you are offering something for public download, or whether you want to expose private content to a defined set of users.



Developing for OpenWrt? Bintray Has an Opkg For You

Opkg280x215OpenWrt is typically associated with network routers and similar equipment, and indeed the official OpenWrt website lists overs 1200 devices on which it runs. Network routers may sound boring (well, to some of us), but as the Internet of Things (IoT) continues its crusade around the world, many people don’t know that OpenWrt is also used on other devices such as phones, robots, sensor networks and more. To accommodate these more glamorous devices, and implement their cool features and functionality, and to give a network router an edge beyond the basic functions, you need a bit more than the Linux kernel on which OpenWrt is based. This good stuff comes in the form of over 3,500 Opkg packages which are available for download from public repositories. The good news is that JFrog Artifactory and JFrog Bintray together offer developers an end-to-end solution to manage their Opkg packages from development through to distributing them automatically to all those routers and robots alike.

While you can use Bintray to distribute your Opkg packages independently of Artifactory, using Artifactory for your OpenWrt development, can speed up your development cycles. For example, Artifactory support for Opkg gives you local repositories for the components you are developing, remote repositories to proxy remote ipk packages and get all those dependencies you need, and it even handles GPG signatures to verify packages. Once your packages are ready to go, use Artifactory’s Distribution Repository and push them to Bintray.

Bintray support for Opkg lets you work transparently with the Opkg client to resolve packages from Bintray and to upload them for distribution. As a mature and universal cloud platform for the distribution of software, Bintray can deliver software updates to users and devices alike over fast CDN, with advanced security features like access entitlements, detailed download stats and logs and more.

Uploading files is easy as 1-2-Oh-there-isn’t-even-a-3

  1. Create an Opkg repository and package in Bintray
  2. Upload your “ipk” file using Bintray’s REST API.
    For example, to upload kool-bot-utils to my robot-utils package in repository opkg-distributions with cURL, I would use:
curl -T kool-bot-utils.ipk -ujaycroaker:myapikey https://api.bintray.com/content/jaycroaker/opkg-distributions/robot-utils/1.0/.

Once your file is uploaded, Bintray will calculate the repository-wide Opkg metadata to maintain compatibility with the Opkg client.

If the version specified in the command don’t exist, Bintray will create it for you. And of course, you can do all of this through Bintray’s REST API, or through the JFrog CLI and in that case, you can do the above and publish the file in a single call.

Downloading is also a snap

The Opkg client works transparently with Bintray. You just need to define Bintray as a feed in your opkg.config file:

echo 'src bintray-feeds https://dl.bintray.com/jaycroaker/opkg-distributions/.' >> /etc/opkg/opkg.conf

Now you just install …

opkg install robot-utils

And if you need authenticated access to the repository, you can just add your Bintray credentials to the config file:

echo 'option http_auth jaycroaker:myapikey' >> /etc/opkg/opkg.conf

You don’t even need to memorize these lines. Bintray has a cool Set Me Up feature for every repository type that displays these code snippets for you to copy and paste into your scripts.


So if routers, robots, and OpenWrt are what make your day, have a look at Bintray’s support for Opkg. Need private repositories? Check out Bintray Premium where you can use signed URLs and access entitlements for fine-grained control over who (or what) can download your ipk files. We envision the day when, not only routers and robots, but all devices get their software updates from Bintray.


Geo Restriction Lets You Control Where Your Content Can Be Downloaded From

geolocation1When you publish software for distribution on Bintray, you generally want to give it as wide a reach as possible and expose it to as many potential users as you can. But there are also cases in which you have to restrict access to certain countries. For example, government security or export regulations may forbid you from doing business with certain countries, or your own company may even restrict your particular organization to distributing your software in your own local region of the world. This is where Bintray’s Geo Restriction feature saves the day.

White lists and black lists

Bintray’s Geo Restriction feature works by letting you define a whitelist or a blacklist per repository. If you create a whitelist, only users from countries entered in the list will be able to download software from the repository. A black list works the opposite way; users from countries in the list are forbidden from downloading software. Bintray determines where a download request is coming from by passing the request IP to a GeoIP provider who determines the country from which the request originated. Then, if a Geo Restriction is applied to the target repository, Bintray will make sure the request can be honored according to the whitelist or blacklist defined. For download requests from countries restricted from accessing a repository, Bintray will respond with an error.

Geo Restriction is available for repositories in an Enterprise account. Gold account users can also see how this feature works, but are limited to one country in the list.

It’s so easy

You can define a whitelist or blacklist by selecting the Geo Restriction tab in the Edit Repository page for each of your repositories. Note that you can only edit the repository if you own it, or if it’s owned by an organization of which you are a member and have the required authorization to do so.


Geo restrictions come first

A Geo Restriction overrides any other form of “permission” in Bintray. In other words, even if a user is a member of your team and has “permission” to download a file, or even if you’ve provided someone with a signed URL or an entitlement  to download a file, if their request comes from a restricted country, it will be blocked.



Keep Your Secrets Safe by Serving Encrypted Files

encrypted_280x215aOnce you have uploaded your content into Bintray private repositories, it’s pretty safe through Bintray’s management of users, organizations, and teams. But what happens when you need to send a private file to someone else? Signed URLs give you an easy way to do that. Just generate your URL signing key and use the URL signing API call to create the URL. You can even make the URL valid for only a limited period of time. Now all you need to do is send the URL to the right person.

But what happens if the URL or the downloaded file gets into the wrong hands? Suddenly the file containing a license key/sensitive company information/anything-else-you-badly-wanted-to-keep-private is exposed..

Well, worry no more.  Bintray can now protect you from that kind of unwanted exposure.

On-the-fly file encryption

Bintray can now encrypt your file, on-the-fly, before it is downloaded through the signed URL. It works like this:

  • Upload your file (either manually through the website or using the REST API) into one of your private repositories.
  • Now you can generate the signed URL which will download the file.
    Nothing new yet, but here’s the thing.
    Now you can also attach a secret to the signed URL and tell Bintray to encrypt the file before it is downloaded. Bintray uses AES 256 CBC encryption for which you can provide your own secret key or ask Bintray to create one for you.
  • Now you can provide your customer with your secret through an alternative communications channel.
  • Once your customer has downloaded the file, they can use the secret key to decrypt the file using an OpenSSL command line. In other words, they need both the signed URL and the secret key.

Let’s See It In Action

Let’s assume we have a top secret “message.txt” file in a repo. We’ll use the REST API to create the signed URL:

curl -X POST -H "Content-Type: application/json" https://bintray.com/api/v1/signed_url/bintray/repo/message.txt?encrypt=true --data '{"valid_for_secs": 3600, "secret" : "my secret"}'

200 OK
"url": "https://dl.bintray.com/bintray/repo/message.txt?expiry=1461681694199&id=fHObRX27TU3EEpjQYIj9HFhbthU%2BPkkD4iXkRibKBYQvTFqlXP9tPUGlk5Xm3qE3&signature=nDt3t7Wj%2BP1B6chQ3AEt8iwaeowWuK66eQTP2tvgfa3AfmSAvR3IrPuVkP3tx90ihT2vMT9m1trJQ6IJZiVdSA%3D%3D"

If you don’t supply a secret, Bintray will create one for you and return it as an Http header: X-Bintray-Secret. Note that for that extra bit of security we limited the validity of the URL to 3600 seconds.

Your customer can now download the file from the returned URL:

curl -o message.txt "https://dl.bintray.com/bintray/repo/message.txt?expiry=1461681694199&id=fHObRX27TU3EEpjQYIj9HFhbthU%2BPkkD4iXkRibKBYQvTFqlXP9tPUGlk5Xm3qE3&signature=nDt3t7Wj%2BP1B6chQ3AEt8iwaeowWuK66eQTP2tvgfa3AfmSAvR3IrPuVkP3tx90ihT2vMT9m1trJQ6IJZiVdSA%3D%3D"

You can choose any secured way to send the user the secret. For example: phone or text message.

Now, to decrypt the file, your customer can use OpenSSL:

$ openssl enc -aes-256-cbc -d -in <encrypted file name> -out <decrypted file name output>

The tool will prompt your customer for the secret and then proceed to decrypt the file.

That’s how you keep your secrets safe. Enjoy!