Time is of the essence: Make an impact using Firehose Events

FirehoseFeatureHave you ever experienced the drive you get when you act on a sudden opportunity that really makes a valuable impact on something? On the other hand, have you ever experienced the disappointment of a missed opportunity?

Let’s face it, every user action can be translated into a short window of opportunity that requires our attention, otherwise, it will be lost forever. Every department in an organization is responsible for ensuring the best possible user experience, and many times we only get one chance to create an impactful initial impression that really makes the difference. So, how can we do this? One of the ways to do this is using JFrog Bintray’s Firehose Events.

We are constantly tracking, measuring and analyzing our user’s behaviors. Most of the time we look back at the user activity that has already happened and try to figure out where we can make improvements. For example, we can measure the number of downloads for any given time, and get valuable insights into the usage of our repositories using JFrog Bintray’s Premium Dashboard. We can even use Live Logs to get detailed stats, including a live download feed as it happens. But what about looking at what’s happening right now, in real-time

What are firehose events?

Bintray’s Firehose Events API enables us to receive live notifications (event triggers / user actions) for a variety of interactions with repositories, and respond to them in real-time with automated activities. Here are the events that you can register for, and some interesting things you can do with them:

  • Successful and failed logins: Repeated failed login attempts for scoped users can trigger an email alert to administrators
  • Downloading a file/artifact: Successful downloads can trigger an engagement email
  • Uploading a file/artifact: Very large uploads can trigger an email alert
  • Usage thresholds: Approaching the storage limit for your repositories can trigger an instant message
  • Deleting a file/artifact: Deleted packages can trigger an alert

Wrapping up with JFrog CLI

Yet again, JFrog CLI provides a convenient and simple interface for your automation scripts. It wraps the Firehose Events API and connects to an Event Notification Stream offering two main advantages in doing so:

  • Automatic reconnect: If the connection to Bintray is lost, JFrog CLI will automatically reconnect you to make sure that none of the events are lost.
  • Event filtering: You might not be interested in all the events coming out of the firehose. The CLI lets you filter out events, by event type, letting you focus only on those that are interesting to you.

Let’s take a look at a simple example. We can use the Firehose Events API to post to a Slack channel and provide a live stream of notifications.

To get events posted to a slack channel, all you have to do is first create a Slack channel, then connect to the event notification stream (Firehose), and pipe the stream to the Slack channel using something as simple as cURL calling the Slack API.

This example uses Bash, JFrog CLI, jq, and cURL. First, you’ll need to install JFrog CLI. Then wrap the following snippet within a script, and run it:

# Connect to the event notification stream
./jfrog bt st  --user= --key= --include="download" |
while read line
do
# Extract the path from the whole event response
  path=$(echo $line | jq .path -r)
  curl   -H"Content-Type:application/json" --data "{\"text\":\"File Downloaded: $path\"}"
done

This is just a simple example of how you can react to Firehose API events. The more creative and inventive Bintray users may come up with ideas like running analytics, creating live graphs and dashboards, issuing relevant alerts, and anything else your imagination can conjure up.

Upgrade to Bintray Enterprise!

Securely Onboarding Colleagues through SAML Authentication

bintray_saml280x215.pngOnce you’ve created your Bintray account, getting your colleagues on board with permission-based access to your organization’s content is not always so easy. You want to use the most secure authentication available, so why can’t you use your corporate SAML server to authenticate your users?

The answer is, now you can.

If you configure your Bintray organization with the details of your SAML server, your colleagues can simply log in using their corporate SSO credentials, and they’re automatically included in your organization. So, not only are you using the most secure authentication your organization has to offer, you’ve also made it easy for your colleagues to get on board.interacting_saml_config_blurred

Now, you can make sure each user is assigned the right permissions by adding them to the corresponding Teams through the UI. That takes care of your colleagues and teammates, but what about external contractors or customers to whom you want to give access? They can’t be managed with teams and permissions since they’re not part of your Bintray organization  (and may not have a Bintray account at all). The answer is to use entitlements defined through the REST API giving them access according to a specified scope – these are what we call “scoped users”. More about that in a future post coming soon.

Automated EULA-protected Downloads

BINTRAY EULA280x215One of the great features of a Bintray Enterprise account is the ability to present a EULA when a user downloads one of your Products. That works well for the general case when the downloading user is a real person who can go through the Bintray UI and physically accept the EULA for publicly available content. This is something we discussed in a previous post. But what if you want to be more selective about who can download your content, and you want to limit it to a defined set of people? And what if those people want to download your content using automated scripts? And what if they don’t even have an account on Bintray? Those are a lot of hoops to jump through, but Bintray now offers a solution to this common circus of circumstances.

You can now offer automated EULA-protected downloads through the REST API to any pre-defined set of users, whether they have a Bintray account or not.

Here’s how it works.

  • You create an entitlement which gives access to your product and then you provide the corresponding access key to your user.
  • Your user browses to a specially constructed Bintray URL
  • Bintray displays a login screen where your user enters the access key as the username, and its password
  • Bintray then displays the EULA for your user to accept.

AcceptEULA

From now on, any REST API call that includes the access key you provided to your user will succeed unhindered by the EULA, since the EULA has already been accepted using that key. In other words – automated EULA-protected download. And who gets this privileged access? Not everyone. Only those who are privileged enough to receive the access key you created to protect your content. In other words private, automated EULA-protected download.

Protecting your products with a EULA has never been easier, whether you are offering something for public download, or whether you want to expose private content to a defined set of users.

 

 

Geo Restriction Lets You Control Where Your Content Can Be Downloaded From

geolocation1When you publish software for distribution on Bintray, you generally want to give it as wide a reach as possible and expose it to as many potential users as you can. But there are also cases in which you have to restrict access to certain countries. For example, government security or export regulations may forbid you from doing business with certain countries, or your own company may even restrict your particular organization to distributing your software in your own local region of the world. This is where Bintray’s Geo Restriction feature saves the day.

White lists and black lists

Bintray’s Geo Restriction feature works by letting you define a whitelist or a blacklist per repository. If you create a whitelist, only users from countries entered in the list will be able to download software from the repository. A black list works the opposite way; users from countries in the list are forbidden from downloading software. Bintray determines where a download request is coming from by passing the request IP to a GeoIP provider who determines the country from which the request originated. Then, if a Geo Restriction is applied to the target repository, Bintray will make sure the request can be honored according to the whitelist or blacklist defined. For download requests from countries restricted from accessing a repository, Bintray will respond with an error.

Geo Restriction is available for repositories in an Enterprise account. Gold account users can also see how this feature works, but are limited to one country in the list.

It’s so easy

You can define a whitelist or blacklist by selecting the Geo Restriction tab in the Edit Repository page for each of your repositories. Note that you can only edit the repository if you own it, or if it’s owned by an organization of which you are a member and have the required authorization to do so.

GeoRestriction

Geo restrictions come first

A Geo Restriction overrides any other form of “permission” in Bintray. In other words, even if a user is a member of your team and has “permission” to download a file, or even if you’ve provided someone with a signed URL or an entitlement  to download a file, if their request comes from a restricted country, it will be blocked.

Enjoy!

 

Who needs a EULA if nobody reads it?

bintray_eula_products_blogpostAs long as you’re distributing public open source software, you don’t need a EULA. Just choose from the over 100 open source licenses Bintray offers to get the level of protection you want. Once you move to distributing commercial software, you need a EULA. This is the contract that you make with all of your users when they download your software from Bintray. Without a EULA, you’re exposing yourself to lawsuits and unmitigated copying and redistribution.

Now, we’ve all seen EULAs. Whether they’re entitled “Terms of Service”, “Terms and Conditions”, or “Here’s what you can and can’t do with my software”, essentially, they are the same. They usually pop up early on in the installation sequence of any new piece of commercial software we install. And I can tell you, with 5-nines of certainty that most of us just click “Next” without reading them, and complete the installation. But there is a breed of people who actually DO read EULAs. They are usually sequestered in some corporate enclave, and it’s their job to make sure that the company they represent can accept the contractual obligations you set forth in your EULA. Yes, these are the company lawyers. Without their “OK”, your potential enterprise customers will not use your software.

Now, as smart as lawyers may be (or not), if your EULA is hidden somewhere deep under /etc/this/that/and/the/other, it’s going to be difficult to find. There’s no reason to make this difficult. Bintray gives you an easy way to expose your EULA through Products.

Bintray Products and EULAs

Say you developed your “packageOfTheCentury”, and, naturally, you want to offer it to developers on different platforms. I.e., you might have your debianPackageOfTheCentury, nugetPackageOfTheCenutury, npmPackageOfTheCentury…and so on. Since all these packages represent the same product, just for different packaging formats, you will want to apply the same EULA to all of them. This is where Bintray Products come into play. Products are available with a Gold or Enterprise account, and let you collect all the packages under one roof as a single coherent offering, making it easy for you to manage. It’s also easier for potential users to find your single product rather than searching for all its components separately, and any update you make to any of the constituent packages in your product is automatically reflected as a new version of the product itself. But the biggest benefit is that you only need to assign a single EULA to your product, and it applies to all of its constituent packages. And the beauty of it is that your EULA protects your product, for both authenticated and anonymous users, before it’s even downloaded, rather than being embedded inside the packages. That also means that it’s easy to update online as any of the constituent packages are updated with new versions. 

Creating a Product is easy.

NewProduct

 

And once your product is created, and you’ve entered its basic details, it’s easy to add packages to it.

ProductPackages

You don’t even have to change your product’s version manually; it is automatically derived by Bintray any time the version of one its constituent packages changes. And if you really want to, you can assign a different EULA to different versions of your product as needed.You don’t even have to change your product’s version manually; it is automatically derived by Bintray any time the version of one its constituent packages changes. And if you really want to, you can assign a different EULA to different versions of your product as needed.

Product Versions

 

All About Bintray EULAs

With a Bintray Enterprise account, you can manage all of your EULAs in one place and set a default EULA to be assigned to all new versions of this product that are created.

ProductEULAs

You can create a new EULA at any time using Asciidoc, Markdown or free text.

Create a New EULA

And once you have a EULA assigned to your product, if anyone (even an anonymous user) tries to download a file from one of the packages in your product, they will have to accept your EULA first. In fact, the download URL provided by Bintray for any of the packages in your product will first pop up the EULA and require the user to accept before proceeding with download. That means you can provide a EULA-protected download link from any other app. Just hover over the package in your download list to see it.

Download URL with EULA

 

Why don’t you try this one to see how it works. Here’s what you should see.

Accept EULA

 

By collecting the different packages you are offering under a single product, governed by a single EULA, you’re protecting your interests, making things simpler, and reducing your (and your customers’) legal costs. Can you think of any better way to protect your IP and keep both your lawyers and your customers happy?

Read all about managing products and EULAs.

 

 

Manage your Bintray and GitHub organizations better together

Bintray’s integration with GitHub is now moving to a new level with GitHub organizations! As a Bintray user who is also a GitHub user, you already know that you can import your GitHub repositories, tags, readme’s, and release notes to Bintray. Now you can also import your GitHub organizations, the organization’s repositories, and even keep your GitHub and Bintray organization’s members in sync! This new feature saves time and effort maintaining your organizations and their members across the two platforms.

Here’s how to do that:

Authorize your GitHub account in Bintray

In order to be able to import GitHub entities to Bintray, your GitHub account should be authorized in Bintray. Your GitHub username has to be provided and authorized in the ‘Accounts’ page in your Bintray profile page:

Authorize Github in Bintray

Grant Bintray access to your GitHub organizations

GitHub organizations should be authorized with Bintray, so Bintray is able to access your GitHub organization. Grant Bintray the access by going to your GitHub profile. Under the ‘Applications’ section you will see the GitHub organization. Select the organizations you would like Bintray to be able to access.

GHOrganizationAccess

You can read more about application authorization with GitHub in the Bintray documentation.

Import a GitHub organization

You can import a GitHub organization while you create a new one in Bintray, or to an existing Bintray organization at any time.

Import GitHub organization to a new Bintray organization

When creating a new organization in Bintray you now see a new option to import your organization from GitHub:

Create new organization

If you choose ‘Import from GitHub’, your GitHub organizations, that have not been imported yet, will be displayed for you to choose from:

Select Organization to Import upon new organization creation

Once you make a selection, your GitHub organization is successfully imported to Bintray. Note that at this point, only the organization is imported, without members or repositories.

At this point Bintray offers you shortcuts to the most common options you naturally wish to do now:

Organization was created successfully

I will elaborate on how to sync members to your imported organization, and how to import an organization repositories later on in this post.

Import GitHub organization to an existing Bintray organization

To associate an existing Bintray organization with a GitHub organization, access the ‘Accounts’ section in your Bintray organization’s profile page. Bintray lets you choose from your accessible GitHub organizations:

Select organization to import to an existing organization

Sync members

Bintray Professional accounts can also sync members from a GitHub organization and have membership changes in a GitHub organization automatically synced to the equivalent Bintray organization. To sync members automatically, click on the ‘Sync’ button in the ‘Members’ section of the organization profile page:

Sync members

The sync will generate an invitation in each member’s Bintray mailbox. Once a user approves his membership, he becomes a fully synced member in the Bintray organization.

The following rules apply once your GitHub organization is imported:

  • All GitHub organization members will be members in the corresponding Bintray organization (as long as they are users of both).
  • GitHub teams are now teams in the corresponding Bintray organization.
  • Members’ permissions are also imported: an ‘owner’ in a GitHub organization will be an ‘admin’ in Bintray, a ‘member’ in GitHub stays a ‘member’ in Bintray.
  • Member’s privacy attributes, ‘private’ and ‘public’ in GitHub, are kept as ‘public’ and ‘nonpublic’ in Bintray .

You can keep the members list synced with Github, so any member added to or removed from GitHub in the future will automatically be updated in your Bintray organization. This saves you the worry of maintaining members in both Bintray and GitHub. You can also disable member sync, so that it is a onetime procedure. Members’ syncing can be enabled or disabled at any time.
For a step by step instructions of how to import GitHub organizations and members, please refer to the user manual.

Import a repository

At this point it makes sense to add a repository to your new organization. Importing GitHub organization repositories is now available! (previously, it was only possible to import personal repositories). In order to do so, create a new repository under your imported organization. In the repository page, click on ‘Import from GitHub’:

Repository page import from GitHub
Bintray will display all the GitHub repositories and their release tags under the imported organization:

Import GitHub repositories

Select the repositories and releases you wish to import, and remember that GitHub repositories will be Bintray packages, and GitHub release tags will be versions in Bintray. Note that the import includes the repository structure and not the actual files.

You can read more about importing GitHub repositories here.

If you use both GitHub and Bintray, this cool new feature will save you time and reduce hassle.
Good Luck!

Catch that Millionth Download with Bintray’s New Statistics API

Want to know exactly how many times your packages have been downloaded? Bintray has always given you download statistics through its UI, but now you can also get them for professional repositories via REST API. Detailed statistics on downloads per version over any time frame give you deep insights into how your software releases are consumed.
If you have never used statistics in Bintray, go ahead and check it out in the user guide.

Let’s see an example
Say I want to get the daily number of downloads of ‘myCoolPackage’ from October 1st to October 8th, 2015. This is what the stats look like in the Bintray UI:

myCoolPackage Downloads UI Stats Per Week

According to the chart, ‘myCoolPackage’ was downloaded a total of 147,752 times in that period. We can clearly see that there were downloads every day; there were dips on October 4th (Sunday) and October 8th (the chart was generated on October 8th around midday), and the most popular versions were 1.1.0 and 1.2.0.

Now you can get all this information programmatically using the new REST API. Here’s the “daily downloads” API as it is described in the REST API documentation:

GET /packages/:subject/:repo/:package/stats/time_range_downloads

To get the statistics displayed in the chart above, I would use the following command:

curl -X GET "https://api.bintray.com/packages/tamarUser/Maven/myCoolPackage/stats/time_range_downloads" -u tamarUser:***my-top-secret-api-key*** -H "content-type:application/json" -d “{\"from\":\"2015-10-01T12:08:56.235z\",\"to\":\"2015-10-08T12:08:56.235z\"}"

I get the following response in JSON format:

{
  "from":"2015-10-01T00:00:00.000Z",
  "to":"2015-10-08T23:59:59.999Z",
  "records":[
      {"date":"2015-10-01","downloads":
        [{"version":"1.0.5","count":1939},
         {"version":"1.1.0","count":6950},
         {"version":"1.1.3","count":293},
         {"version":"1.1.7","count":116},
         {"version":"1.2.0","count":10111},
         {"version":"1.2.1","count":1329},
         {"version":"1.2.2","count":1706}]},
      {"date":"2015-10-02","downloads":
        [{"version":"1.0.5","count":315},
         {"version":"1.1.0","count":6975},
         {"version":"1.1.3","count":198},
         {"version":"1.1.7","count":121},
         {"version":"1.2.0","count":9967},
         {"version":"1.2.1","count":1290},
         {"version":"1.2.2","count":1759}]},
      {"date":"2015-10-03","downloads":
      ...]
}

The response provides all of the same data that Bintray uses to create the chart in the UI. For each day within the requested date range in which downloads occurred, it lists the number of downloads per version. As simple JSON output, you can easily parse the response and use it any way that helps you analyze your package downloads quickly and effectively. You are now able to identify trends in downloads, your popular versions and more.

Other statistics REST APIs include: total downloads and downloads by country. Keep an eye on this blog to hear about new APIs when we add them.

Good luck!