[Case Study] Bintray Automation for a Tedious Situation: Automating Package Distribution

Did you ever find yourself spending large amounts of time on a manual process, just wishing there was an automatic way to do it?

This case study will take you through how James Ward, Engineering and Open Source Ambassador at Salesforce.com, developed an automated process using JFrog Bintray for on-demand deployments of WebJars, that are created, distributed and available within the community.

After realizing that his many hours of manually deploying WebJars during his free time were unrealistic to maintain in the long run, Ward chose to use Bintray’s extensive and easy-to-use REST API to automate the process and make it available to the community on demand using JCenter, Bintray’s public Maven repository.

Ward’s WebJars project is a classic example of how JFrog Bintray can be used to scale up a tedious and time-consuming manual procedure into an easy and automated one.

Download the case study to learn more on how James Ward chose Bintray to scale thousands of WebJar Files.

The ABCs of Distributing Android Libraries

ABCDistributionFeatureBintray’s central repositories, JCenter and Conan-Center, are binary hubs for public OSS Maven and Conan (C++) packages respectively. They offer a great channel to distribute your public OSS packages. Having been around for a while, JCenter has become one of the most comprehensive sources for public OSS Maven packages, and is the channel-of-choice for many Maven projects. So we weren’t surprised to discover this blog post that describes how easy it is to distribute Android libraries through Bintray and JCenter.

Distributing Android libraries (or any other Maven package for that matter) through JCenter has its advantages. As one of the most popular Maven hubs, it exposes your package to a huge audience, and yet, you retain full ownership and can control how your package evolves. Basically, you upload your package to one of your public Maven repositories and ask for it to be included in JCenter. Once approved by the Bintray team, your package will be searchable on JCenter and freely available for download. You can learn about the details in the blog or look it up in the Bintray User Manual.

So if you’re ready to give your package world-wide fame, go ahead and open an account on Bintray. Bear in mind that since JCenter is a curated repository with trusted content, you need to be on a fully registered OSS or Premium account (as opposed to being on a trial) before you can link your package.

To start with a free OSS account, you can sign up here.

If you’re ready to go Premium (Pro or Enterprise), go here.

And if you want the full Enterprise experience, without committing to it yet, you can start a free trial (just remember to switch to an OSS or Premium account if you want to use JCenter).

Time is of the essence: Make an impact using Firehose Events

FirehoseFeatureHave you ever experienced the drive you get when you act on a sudden opportunity that really makes a valuable impact on something? On the other hand, have you ever experienced the disappointment of a missed opportunity?

Let’s face it, every user action can be translated into a short window of opportunity that requires our attention, otherwise, it will be lost forever. Every department in an organization is responsible for ensuring the best possible user experience, and many times we only get one chance to create an impactful initial impression that really makes the difference. So, how can we do this? One of the ways to do this is using JFrog Bintray’s Firehose Events.

We are constantly tracking, measuring and analyzing our user’s behaviors. Most of the time we look back at the user activity that has already happened and try to figure out where we can make improvements. For example, we can measure the number of downloads for any given time, and get valuable insights into the usage of our repositories using JFrog Bintray’s Premium Dashboard. We can even use Live Logs to get detailed stats, including a live download feed as it happens. But what about looking at what’s happening right now, in real-time

What are firehose events?

Bintray’s Firehose Events API enables us to receive live notifications (event triggers / user actions) for a variety of interactions with repositories, and respond to them in real-time with automated activities. Here are the events that you can register for, and some interesting things you can do with them:

  • Successful and failed logins: Repeated failed login attempts for scoped users can trigger an email alert to administrators
  • Downloading a file/artifact: Successful downloads can trigger an engagement email
  • Uploading a file/artifact: Very large uploads can trigger an email alert
  • Usage thresholds: Approaching the storage limit for your repositories can trigger an instant message
  • Deleting a file/artifact: Deleted packages can trigger an alert

Wrapping up with JFrog CLI

Yet again, JFrog CLI provides a convenient and simple interface for your automation scripts. It wraps the Firehose Events API and connects to an Event Notification Stream offering two main advantages in doing so:

  • Automatic reconnect: If the connection to Bintray is lost, JFrog CLI will automatically reconnect you to make sure that none of the events are lost.
  • Event filtering: You might not be interested in all the events coming out of the firehose. The CLI lets you filter out events, by event type, letting you focus only on those that are interesting to you.

Let’s take a look at a simple example. We can use the Firehose Events API to post to a Slack channel and provide a live stream of notifications.

To get events posted to a slack channel, all you have to do is first create a Slack channel, then connect to the event notification stream (Firehose), and pipe the stream to the Slack channel using something as simple as cURL calling the Slack API.

This example uses Bash, JFrog CLI, jq, and cURL. First, you’ll need to install JFrog CLI. Then wrap the following snippet within a script, and run it:

# Connect to the event notification stream
./jfrog bt st  --user= --key= --include="download" |
while read line
do
# Extract the path from the whole event response
  path=$(echo $line | jq .path -r)
  curl   -H"Content-Type:application/json" --data "{\"text\":\"File Downloaded: $path\"}"
done

This is just a simple example of how you can react to Firehose API events. The more creative and inventive Bintray users may come up with ideas like running analytics, creating live graphs and dashboards, issuing relevant alerts, and anything else your imagination can conjure up.

Upgrade to Bintray Enterprise!

What Makes a Dashboard “Premium”?

280x215.pngOnce you’ve uploaded a package to Bintray, the one thing you want to see, more than anything else, is downloads. Without downloads, your package is like the proverbial tree that falls down in an empty forest, the proverbial sound of one hand clapping, the … well, you get the picture. Bintray is happy to give you download stats, color-coded per version and all, even for packages in an OSS repository.

null

While a chart like the one above is a great way to start your morning, eventually you find yourself wanting more, especially if you’re an enterprise with many repositories, even more packages, and (hopefully) lots of customers (both internal and the paying kind).

This is where Bintray’s Premium Dashboard steps up to the plate.

Once you go “Premium”, Bintray shows you three different views of usage:

Total Usage

This is identical to the download statistics you got before going Premium.

By Repository

Here’s where things start to get more interesting. You get a breakdown of downloads and storage for each repository in your account. This adds an important dimension to your total usage. If you see peaks of usage (which may add up in your monthly bill), you can now identify exactly which repositories are generating those downloads. This lets you make an informed decision on how to handle those repositories. Perhaps you should change your Bintray plan to include premium repositories or Geo and IP restrictions, or maybe you should just move certain packages to a different repository and distribute usage better. It’s like the difference between just getting the total charge on your phone bill, as opposed to getting an itemized list of calls you made. Once you have the information, you can make informed decisions on what to do.

null

By Business Unit

And here’s where things get really interesting. How can you monitor downloads of your software and storage used per customer; whether it’s a paying customer who pays per download volume or an internal “customer” that you are monitoring for usage? Well, you could assign a separate repository for each customer. You might even think up some neat naming convention for repositories that shows to which customer each repository was assigned. But what if each customer downloads from a number of repositories? Then, you could keep a note of all the repositories assigned to a customer and do the math. That might work, but it might also get out of control when your organization gets big enough with many repositories serving many customers (the problems we all want to have). There’s no need to develop elaborate schemes to handle your billing; Bintray does all the math for you with Business Units. If you define each customer as a Business Unit, you can then assign all the repositories that serve that customer to the corresponding Business Unit, and get usage stats per Business Unit (i.e. per customer).

null

But wait; there’s more

What if one of your customers is downloading too much? “Why is this a problem?” you ask. Well, not all customers are the paying kind. For example, you may have a business unit assigned to a contractor who is doing some work for you, and you need to monitor and limit usage of the repositories you assigned to your contractor. If you’re on an Enterprise plan, Bintray lets you set usage thresholds using the REST API. Through the Usage Thresholds REST API endpoints you can set a threshold on maximum monthly storage, monthly download volume or daily download volume, and get alerted when any of these thresholds is exceeded. The alerts are delivered as firehose events, but you can also specify any number of email addresses that should also receive the alert.

So there’s your answer. A dashboard provides you with information. A premium dashboard gives you different ways to analyze that information giving you valuable insights into usage of your repositories. And Bintray’s premium dashboard also gives you control over how your repositories are used with usage thresholds. You see, there’s “premium”, and then there’s “Bintray Premium”.

Have an account? Go Premium

Don’t have an account? Sign up for a free trial

Securely Onboarding Colleagues through SAML Authentication

bintray_saml280x215.pngOnce you’ve created your Bintray account, getting your colleagues on board with permission-based access to your organization’s content is not always so easy. You want to use the most secure authentication available, so why can’t you use your corporate SAML server to authenticate your users?

The answer is, now you can.

If you configure your Bintray organization with the details of your SAML server, your colleagues can simply log in using their corporate SSO credentials, and they’re automatically included in your organization. So, not only are you using the most secure authentication your organization has to offer, you’ve also made it easy for your colleagues to get on board.interacting_saml_config_blurred

Now, you can make sure each user is assigned the right permissions by adding them to the corresponding Teams through the UI. That takes care of your colleagues and teammates, but what about external contractors or customers to whom you want to give access? They can’t be managed with teams and permissions since they’re not part of your Bintray organization  (and may not have a Bintray account at all). The answer is to use entitlements defined through the REST API giving them access according to a specified scope – these are what we call “scoped users”. More about that in a future post coming soon.

IP Restriction with White CIDR and Black CIDR

IP restrictionImagine this scenario. Your flagship product is doing OK; you’re getting downloads. Nevertheless, increasing sales is always a top priority, so you decide to create a free OSS version to boost usage and generate more awareness in the market. It’s also a great product, free to download, and is a great teaser for the upsell to your commercial version. So you upload the OSS version to your organization’s public repository on Bintray. After a while, you check its stats and are delighted to see thousands of downloads, and yet, there’s no buzz. If so many people are downloading your package, why is nobody talking about it? After checking your logs, you discover that most of those downloads you were so happy about are from the same set of IP addresses.

Someone is reverse-spamming you, and you need to block them.

Geo-Restriction could do the job, but then you don’t want to restrict whole countries at a time. You need something more targeted. Bintray’s new capability for IP restriction will do the job.

IP restriction is very similar to geo-restriction, only instead of defining your whitelist and blacklist as countries, you define them as a range of IP addresses using CIDRs. This gives you much more control letting you allow or block download at any level of granularity down to a single specific IP address. Whatever IP those bots are operating from can now be easily blocked so you can get the real download statistics you were interested in.

You can define your CIDR whitelist and blacklist through Bintray’s REST API or in the Edit Repository page.

IP restriction

Note that if there are any overlapping IPs, (i.e. IP addresses that are included in CIDRs that are in both the whitelist and the blacklist), then blacklisting them takes precedence. And if you accidentally list the exact same CIDR in both the whitelist AND the blacklist, Bintray will catch that and issue an error.

Ready to use IP restrictions? Get a quote for an enterprise account and ask for a free trial.

Automated EULA-protected Downloads

BINTRAY EULA280x215One of the great features of a Bintray Enterprise account is the ability to present a EULA when a user downloads one of your Products. That works well for the general case when the downloading user is a real person who can go through the Bintray UI and physically accept the EULA for publicly available content. This is something we discussed in a previous post. But what if you want to be more selective about who can download your content, and you want to limit it to a defined set of people? And what if those people want to download your content using automated scripts? And what if they don’t even have an account on Bintray? Those are a lot of hoops to jump through, but Bintray now offers a solution to this common circus of circumstances.

You can now offer automated EULA-protected downloads through the REST API to any pre-defined set of users, whether they have a Bintray account or not.

Here’s how it works.

  • You create an entitlement which gives access to your product and then you provide the corresponding access key to your user.
  • Your user browses to a specially constructed Bintray URL
  • Bintray displays a login screen where your user enters the access key as the username, and its password
  • Bintray then displays the EULA for your user to accept.

AcceptEULA

From now on, any REST API call that includes the access key you provided to your user will succeed unhindered by the EULA, since the EULA has already been accepted using that key. In other words – automated EULA-protected download. And who gets this privileged access? Not everyone. Only those who are privileged enough to receive the access key you created to protect your content. In other words private, automated EULA-protected download.

Protecting your products with a EULA has never been easier, whether you are offering something for public download, or whether you want to expose private content to a defined set of users.