Intro
I recently started a new, grander, project for my spare time. The project involves working with Podcast feeds, and I was going to use this as an opportunity to use a framework I haven’t before, Dropwizard. I found a Java library that did what I needed in MarkusLewis – Podcast Feed Library; except this library only read feeds, I want to be able to read AND write. I decided to make my own, and I wanted to host the library, allowing anyone else to use it if they want. I created the repo and got a basic version working. This is the repo which can be referenced as an example.
I am using GitHub Actions as my CI/CD pipeline. I thought I should easily be able to host the final Jar files there for Gradle. Turns out, this is sort of true… If you host your library on GitHub itself, as this doc goes over, you can easily upload and host the packages; except there is no un-authenticated access to it. No matter what, an end user has to auth with Gradle/Maven before downloading the assets. Instead of dealing with that (specially for a public repo), I thought I would give a try to getting my package into Maven Central. Once I figured out the process, and found out how to publish with up to date Gradle, it as straight forward. I thought I would document it for the greater internet, and my future self (I have already used it). I know others have done this as well, except I wanted to do it with Gradle instead of Maven, with GitHub Actions doing all the work.
Throughout this guide there are items you need to record to bring to the next step, I have underlined the important ones.
Steps:
- Setup Repo
- Register For Maven/Sonatype
- Setup GPG for Repo
- Configure GitHub
- Publishing
Setup Repo
Setup a normal GitHub repo, and setup a blank Gradle project. More on the repo/Gradle config later.
Register for Maven/Sonatype
Sonatype is the company who runs Maven Central. They allow free hosting and registration for Java Libraries; the main requirement is for it to be under 1GB in size per file. This adventure starts over at their Jira to register for an account, this Jira account will be your credentials for all future interactions with Maven Central, so make them secure, and have a long password! Once you have an account, use the above like to go to their Jira again to create an issue. This ticket grants you permissions to publish to https://s01.oss.sonatype.org/. You can also login there with the credentials created for Jira. You will have to verify either your GitHub account, or your domain before publishing. A bot handles all this and I had it done in 20 minutes or so. I have a domain I wanted to use, and there is a guide on how to go through this process.
Once you register a group ID you can use this account to publish anything under that ID; for example, I registered my domain of ntbl.co (making the group ID in Java terms “co.ntbl”. First, I published the library above, then I added a fork of a Java-Lame library; I tried to submit a ticket for the second library to be sure, and the bot tells you that you are already good to go.
Setup GPG for the Repo
One requirement for posting assets to Maven Central is to GPG sign the packages. This means we need to generate a key, and then upload the private portion of the key to GitHub secrets, and the public to a public key repository. Below are the commands to do this, the key ID is an example one I have, you will need to replace it with yours:
gpg --gen-key
gpg --list-keys
gpg --export --export-options backup --output public.gpg co.ntbl.podcastfeedhandler
gpg --export-secret-keys --export-options backup --output private.gpg co.ntbl.podcastfeedhandler
gpg --export-ownertrust > trust.gpg
gpg --list-secret-keys --keyid-format LONG
gpg --export-secret-keys --armor 3F6F38BA13BEBB6941F823DCEFAAE414FF016215
gpg --keyserver keys.openpgp.org --send-keys 3F6F38BA13BEBB6941F823DCEFAAE414FF016215
Line 1 creates the keys, for name you enter the full project name, for example: co.ntbl.podcastfeedhandler . Group ID + the project root name. The email can be any email you have. Then the passphrase which will be used and uploaded to Github secrets. I suggest using a password generator and making it long, you should never have to actually type this in. Next, the exports are for you to back up the key incase the system you are creating it on dies and the data is lost in the GPG instance. You shouldn’t generally need it after this is setup, but it felt like best practice.
Record the output of the 7th line (export-secret-keys), that will need to be added to the Github secrets in the next step.
Configuring GitHub
The last command publishes the public keys to a global repo which is checked against. If this publish is not done, then the verification of the package will fail.
The two items we need to upload to GitHub for GPG are the password added when the key was generated, and the private key we got from the 7th command.
Go to your GitHub repo, then go to the Settings tab. Using the left-hand navigation, go to “Secrets and variables”, and the “Actions” submenu.
We need to create 4 secrets; these need to be kept secret:
GPG_SIGNING_KEY– The private key, copy the text from the “–export-secret-keys” command, this formats it correctly. The string should start with “—–BEGIN PGP PRIVATE KEY BLOCK—–“GPG_SIGNING_PASSPHRASE– The password added when generating the keyOSSRH_TOKEN– This is the password you set for Sonatypes JiraOSSRH_USERNAME– The Sonatype Jira username
Below is a minimal example build.gradle for your project. I removed a lot of normal extra things you would add to a buidl.gradle, to see a full example, visit this GitHub repo.
Gradle
plugins {
id 'java-library'
id 'signing'
id 'maven-publish'
}
group = 'co.ntbl'
version = '0.1.2-SNAPSHOT'
rootProject.description = 'Read and Write Podcast feeds from Java.'
sourceCompatibility = 11
targetCompatibility = 11
tasks.register('createProperties') {
doLast {
new File("$projectDir/src/main/resources/version.properties").withWriter { w ->
Properties p = new Properties()
p['version'] = project.version.toString()
p.store w, null
}
}
}
classes {
dependsOn createProperties
}
jar {
manifest {
attributes(
"Class-Path": "co.ntbl.podcastfeedhandler",
"Main-Class": "PodcastFeedHandler",
"Implementation-Title": project.name,
"Implementation-Version": version,
"Implementation-Vendor": "Daniel Berkowitz",
"Build-Jdk": org.gradle.internal.jvm.Jvm.current(),
"Gradle-Version": GradleVersion.current().toString()
)
}
duplicatesStrategy = DuplicatesStrategy.EXCLUDE
from {
configurations.runtimeClasspath.collect { it.isDirectory() ? it : zipTree(it) }
}
}
java {
withJavadocJar()
withSourcesJar()
}
ext.admin = System.getenv("MAVEN_USERNAME")
signing {
required { admin }
def signingKey = System.getenv("GPG_SIGNING_KEY")
def signingPassword = System.getenv("GPG_SIGNING_PASSPHRASE")
useInMemoryPgpKeys(signingKey, signingPassword)
sign publishing.publications
}
repositories {
mavenCentral()
}
dependencies {
...
}
//
// MAVEN
//
publishing {
publications {
mavenJava(MavenPublication) {
from components.java
pom {
name = 'PodcastFeedHandler'
description = rootProject.description
url = 'https://github.com/daberkow/PodcastFeedHandler'
licenses {
license {
name = 'MIT License'
url = 'https://github.com/daberkow/PodcastFeedHandler/blob/main/LICENSE'
distribution = 'repo'
}
}
developers {
developer {
id = 'daberkow'
name = 'Daniel Berkowitz'
email = 'dansberkowitz@gmail.com'
}
}
scm {
connection = 'scm:git:git://github.com/daberkow/PodcastFeedHandler.git'
developerConnection = 'scm:git:ssh://git@github.com:daberkow/PodcastFeedHandler.git'
url = 'https://github.com/daberkow/PodcastFeedHandler'
}
}
}
}
repositories {
maven {
name = "OSSRH"
if (admin) {
credentials {
username = System.getenv("MAVEN_USERNAME")
password = System.getenv("MAVEN_PASSWORD")
}
}
def releasesRepoUrl = 'https://s01.oss.sonatype.org/service/local/staging/deploy/maven2/'
def snapshotsRepoUrl = 'https://s01.oss.sonatype.org/content/repositories/snapshots/'
url = version.endsWith('SNAPSHOT') ? snapshotsRepoUrl : releasesRepoUrl
}
}
}
A few things to point out. Under publishing you need to enter all the information for this repository/project. If you have another publishing section in your Gradle file you will need to condense them together. Having multiple leads to Gradle getting confused and usually using the first one it sees. You will also see some variables such as “MAVEN_USERNAME”, these get the values of our secrets during the GitHub actions publish process, which we will go over next. I am getting the version, and using the end of it containing “SNAPSHOT” to say if we should publish to a snapshot repo or prod.
I also am using the build.gradle version as the canonical version. This variable could be in a Gradle settings file, or properties, but for ease I have it in the build file. I want 1 version file location; having multiple leads to more confusion during releases. The createProperties task creates a properties file that is added to the build to give the code itself a way to see which version it is. There are more elaborate ways to do this, but it works for me. This function does need the resources folder to be in the “src/main” folder; if your project is not using this the easiest way to add an “empty folder” is add the “resources” folder and then add the following .gitignore to it. This will make sure the contents of this folder are never saved.
# Ignore everything in this directory
*
# Except this file
!.gitignore
Requirements for posting to Maven central are: including source, checksums, Javadocs, and signing your packages. I am using useInMemoryPgpKeys to sign in GitHub Actions. This is part of the signing plugin. I have seen others use sign configuration.packages instead of sign publishing.publications, I found that not to work in many trials.
GitHub Actions
In your repository, create a .github folder, then a workflows folder. Below is my publish.yml, or it is available here. This file is currently set to publish when a new release is tagged, you can also change this to commits or some other trigger.
name: Publish package to the Maven Central Repository and GitHub Packages
on:
release:
types: [published]
jobs:
publish-release:
runs-on: ubuntu-latest
steps:
- name: Checkout latest code
uses: actions/checkout@v3
- name: Set up JDK 11
uses: actions/setup-java@v3
with:
distribution: adopt
java-version: 11
- name: Validate Gradle wrapper
uses: gradle/wrapper-validation-action@e6e38bacfdf1a337459f332974bb2327a31aaf4b
- name: Publish package
uses: gradle/gradle-build-action@67421db6bd0bf253fb4bd25b31ebb98943c375e1
with:
arguments: publish
env:
MAVEN_USERNAME: ${{ secrets.OSSRH_USERNAME }}
MAVEN_PASSWORD: ${{ secrets.OSSRH_TOKEN }}
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
GPG_SIGNING_KEY: ${{ secrets.GPG_SIGNING_KEY }}
GPG_SIGNING_PASSPHRASE: ${{ secrets.GPG_SIGNING_PASSPHRASE }}
Here we convert GitHub secrets to local environment variables. Note the change in name from OSSRH_USERNAME to MAVEN_USERNAME and OSSRH_TOKEN to MAVEN_PASSWORD. This is simply to make the variables more clear, and they can be whatever you wish. We also validate Gradle for this final build. Another note, in my setup we are not passing assets from earlier builds into this publish stage, we are rebuilding the jar completely, depending on the size of your job, this may or may not make sense. If you have all this setup correctly, you should be able to commit the code, tag a release with “0.0.1-SNAPSHOT” or any version ending in SNAPSHOT and it should publish to the snapshot repo.
Publishing
Now that we have working snapshot releases, we need to do a full release. This involves you using the credentials created with the Sonatype Jira account earlier and logging into the Nexus panel. When you are ready, go to GitHub, and mark a new release with the version not ending with SNAPSHOT. The GitHub action should finish successfully, yet your asset is not up at https://repo1.maven.org/maven2/ yet. Head over to https://s01.oss.sonatype.org/ and click “Log In” in the top right.
Select “Staged Repositories” on the left. Note: this server seems to be very busy during the day, doubly so if it is a weekday. You will frequently see “There was an error communicating with the server: request timed out”. Come back later or keep hitting refresh.
Clicking a repository will allow you to browse the contents, and make sure it looks how you want it to. When you are ready you click “Close” at the top of the pane to finalize this version. Closing the repository starts all the checks on the repository, this includes making sure GPG signatures are there, the sources, Javadoc, and checksums are there. If they are not, you will get an error and be forced to Drop the release and try again. You also will get a vulnerability scan, including dependencies, to your email on file.
After the repo successfully closed, you can click Release! This is another stage where you can get many timeouts and be forced to wait till the server is less busy. After it successfully releases, it takes about 30 minutes for it to show up in the global Maven repo.
Selecting “Repositories” at the left allows you to browser the global Snapshots and Releases repositories; I have found this screen updates quicker than other locations to see if your assets are starting to propagate, including faster than the main Maven repo.
After about 30 minutes, your release should start to show up at Maven Search, although it can take longer. Another popular place to check packages is mvnrepository, I have found this site seems to take about a day to find new packages.
I hope this guide can help someone (and probably my future self), feel free to drop a comment if it helps or if something is unclear!
BuildingTents 