Quick Blurb

SSSD with Active Directory Only Showing Primary Group

I was domain joining some Redhat Enterprise Linux 7 boxes to a Windows domain. Everything went smoothly except many of my users could only see their Primary groups. Some users whom had more permissions on the domain could see all their groups, just not some particular users. This seems to be a common failure scenario for SSSD with AD, and many people have opened bugs or chimed in with different fixes online. I found the solution on one forum post, and it saved me, and I wanted to amplify it.

As long as some of your users can see all their groups, you know its not exactly a problem with RHEL connecting to AD, or a protocol like LDAP being blocked. A odd side effect of this setup was periodically the groups could be scanned and then it would show the users in that group. If I ran “sss_cache -E“, then “getent group SecondaryGroup“, some of the time it would show the users inside the group. Then once the user logged in, the user would be removed via that command, as well as when I ran “groups” under the user.

The SSSD log didnt have a ton of help other than it couldn’t read all the groups. I tried a TON of the recommended settings, like enabling enumerate = True, enumerate = false, ldap_use_tokengroups = true, ldap_use_tokengroups = false; none of these changed anything. Then https://serverfault.com/a/938893 mentioned it may be a permissions problem between the computer object in AD and the user object. I looked and sure enough, my system had NO permissions on the users that were failing. I attempted to add the tokenGroups┬ápermission mentioned in this article and that still didnt help, but we were on the right track!

The answer came from https://serverfault.com/a/796005, there is a permission needed called “Read Remote Access Information”, once that is granted to your computer object onto the user, then secondary groups will start populating. I gave “Domain Computers” that permission, since it seemed to only be effecting some of the Linux systems and Windows was happy to have it as well.

Some random commands that can help you debugging SSSD:

SSSD likes to cache a lot, making it hard to troubleshoot, using the following clears all caches and restarts SSSD:

systemctl stop sssd && rm -rf /var/lib/sss/db/* && rm -rf /var/lib/sss/mc/* && systemctl start sssd

CentOS/Rhel 8 Auto login Fix

I have a PXE environment that requires systems to boot up, then automatically login and start a program on boot. All of a sudden this stopped working after years of working. It took me a while to figure it out so figured I would post in case anyone else ran into this.

I have been doing auto login the recommended systemd for a while, as shown: https://wiki.archlinux.org/title/Getty. I copied /lib/systemd/system/getty@.service into /etc/systemd/system/getty@tty1.service. Then with a script edited it using sed in the build pipeline. In the end the line was:

ExecStart=-/usr/bin/agetty --noclear %I $TERM --autologin username

This worked for YEARS, then suddenly stopped. In investigating, I saw another file was being written next to mine at /etc/systemd/system/getty@tty1.servicee ; with another e added to the end of service, making it servicee. After a lot of playing around with it and looking at other guides I figured out, there was a update to systemd/getty and now it cares that all options are before the terminal variable is presented. Changing that line to the following fixed it.

ExecStart=-/usr/bin/agetty --noclear --autologin username %I $TERM 

Booting Dell Precision 7910 and Assassins Creed Valhalla

I recently was setting up a Dell Precision 7910 at work that was in storage for a while. I could not get the system to POST at all. I would give some power on lights, but then sit there and eventually turn off. I searched online and didn’t see any reference to this, so I wanted to make a quick note for the internet. In the system there is a big plastic air guide, that air guide holds the CPU fan. If the cable for the CPU fan gets unplugged, the system will not post.

On a completely unrelated note, I recently got Assassins Creed Valhalla and was enjoying it except every major cut scene or sometimes between areas the game would crash. This was leading to several crashes a hour. No one online seemed to have such a bad time. Long story short, I realized the game was also having issues with cloud saves. It turns out the game frequently attempts to reach out to Ubisoft servers, and instead of failing gracefully, hard crashes. I happen to have a bunch of the Pi-Hole lists in my firewall (Firebog seemed to be the culprit) and the following domains kept being blocked:

*.ubi.com
ubiservices.data.ubi.com

I believe only the “ubiservices.data.ubi.com” domain is the one that needs to be allowed, but to be save I just allow-listed the whole ubi.com domain. Since then the game has not crashed once. Maybe their developers should learn about try/catch blocks…

Quick Update

I have not written in a bit so I thought I would give a quick update. I haven’t had a lot of time recently to work on things because I was moving, and had to re-certify for Cisco. Cisco gave everyone 6 months additional time on certificates because of Covid last year (yay), but then I saw a reddit post that mentioned they had no extended the time between tests for multi test certs (booo). With a bit over a month left of time I studied for the CCNP Core Security cert and was able to pass. Now I have a CCNP “Enterprise” (the old Route and Switch) and a CCNP Security. The new Cisco cert system is very odd with each test being a “specialty” and then combinations of different tests adding up to other certs.

I have continued to play with the Mister FPGA. I never had an Amiga and there are some fun games for that system. A lot of the other games I have been laying on it are from my childhood, running the ao486 core. A bit ago I got another retro computer kit that will eventually be added to the list, but this one is a bit more involved. It is a full replica IBM 5170 motherboard. It needs soldered together, as well as add on cards found for it. Hopefully I will have more time for projects in the upcoming months, and at the same time I will try to do some more documentation of the current homelab.

Cisco ISR 4451 Serial Password Recovery

I had to password recover a Cisco ISR 4451, and kept having issues getting into the ROMMON prompt. Every guide mentioned sending a BREAK character during startup, but I could not get that to work. I was using the mini-USB port in the front, and as far as I knew did not have password recovery disabled. It turns out there is a problem with the mini-USB port and the Mac driver, I switched to using a traditional serial cable with a DB-9 connector/RJ45 serial port and suddenly I could get into ROMMON. I wanted to post incase anyone else runs into this.

Below is the startup process, at the end there you should be able to send a BREAK character.

Initializing Hardware ...

System integrity status: 00000610
Rom image verified correctly


System Bootstrap, Version 15.3(3r)S1, RELEASE SOFTWARE
Copyright (c) 1994-2013  by cisco Systems, Inc.

Current image running: Boot ROM0

Last reset cause: PowerOn
Cisco ISR4451-X/K9 platform with 4194304 Kbytes of main memory


Warning: filesystem is not clean
File size is 0x1d482044
Located isr4400-universalk9.03.16.04b.S.155-3.S4b-ext.SPA.bin 
<SEND BREAK HERE>

Using a Custom User-Agent with Google OAuth Client in Java

I have been using the Google OAuth for some of my projects at work for a while. A recent request was to add custom user-agent strings to different apps for the people doing analytics on which apps are using the authentication servers. I have some functions that do custom HTTP Get calls using the Bearer token we get from the OAuth flow, then the library also does its own calls behind the scene. I was able to add a user-agent to my calls easily, but the under the hood ones the library does kept coming up as “Google-HTTP-Java-Client/1.34.2 (gzip)”. I tried a few different ways, and at the same time was searching online, and didn’t see anyone speaking about this. Below is a quick block to put into your app if you want to set the user-agent.

These are the current versions of the OAuth library, and the http client I have been using to do auth.

compile group: 'com.google.oauth-client', name: 'google-oauth-client', version: '1.31.4'
compile group: 'com.google.oauth-client', name: 'google-oauth-client-servlet', version: '1.31.4'
compile group: 'com.google.http-client', name: 'google-http-client', version: '1.39.0'
compile group: 'com.google.http-client', name: 'google-http-client-jackson2', version: '1.39.0'

For my setup, I have the OAuth Servlet that initializes the OAuth flow, then a second servlet which handles the callback; as documented here. I added to the “class OauthCallback extends AbstractAuthorizationCodeCallbackServlet” the following ConnectionFactory under the override for the initializeFlow() function. Replace “myApp-v1.0.1” with your app name. Hope this helps someone!

@Override
protected final AuthorizationCodeFlow initializeFlow() throws IOException {
    ConnectionFactory connectionFactory = url -> {
        HttpURLConnection httpURLConnection = (HttpURLConnection) url.openConnection();
        httpURLConnection.setRequestProperty("user-agent", "myApp-v1.0.1");
        return httpURLConnection;
    };
    return new AuthorizationCodeFlow.Builder(BearerToken.authorizationHeaderAccessMethod(),
            new NetHttpTransport.Builder().setConnectionFactory(connectionFactory).build(),
            new JacksonFactory(),
            .... (code removed);
}

ESXi Migration & Lenovo ThinkCentre M710s

I have started a transition from Hyper-V and Storage Spaces Direct to VMWare vSphere and vSAN. I apologize that these blog posts order is all over the place. Part of the transition is upgrading the hardware on some of the hosts I have, including getting 250GB NVME drives for vSAN cache. I started the migration with one of the desktops that run in the cluster, a Lenovo ThinkCentre M710s. After finding the small slot the NVME drive goes in, I realized there is a manufacture piece of plastic you are supposed to get to install a NVME drive. Since I do not have that, and do not want to pay for it, I spent a good bit more than a hour the first day of the migration creating this bracket and 3D printing it. Then while that was printing, I realized one of the feet on the system had gone missing, so I made a small one of those.

This post is just a quick update and a preview of more to come.

NVME Drive Holder: Lenovo ThinkCentre M710s NVME Bracket by danberk – Thingiverse

Foot: Lenovo ThinkCentre M710s Foot by danberk – Thingiverse

Booting VMware vSphere ESXi 7.0 on Certain Dell Hardware

I recently attempted to boot a Dell Precision M6800 into ESXi 7.0u1 to test some functionality before going to prod. Unfortunately this was met with “Invalid Partition Table”, switching between UEFI and BIOS boot didn’t seem to fix it giving “No boot device available” instead. After searching online I found this, https://communities.vmware.com/t5/ESXi-Discussions/quot-Invalid-Partion-Table-quot-Error-booting-ESXi-7-from-USB/m-p/1823852 which had comments such as “just dont run on a laptop” which was not very helpful.
I spent a chunk of time playing with the partitions and seeing how they were configured. I noticed when I went into the UEFI on the laptop it said it couldn’t find any file systems available, but when I loaded Windows or Linux on the system, the UEFI could see those boot partitions. I tried updating the firmware like Dell recommended, with no change. I then realized the ESXi 7.0 image is FAT16 for the EFI partition, while all other EFI partitions I have seen are FAT32.

I copied the files and folder out of the boot partition, reformatted it with FAT32 instead of FAT16, marked it as EFI type (ESP in Gparted), and moved the files back. The system booted fine the first time, with ESXi running happily. If you need boot ESXi on a Dell M6800, or M4800, or other give that a try. If this worked or didn’t work for you leave a comment below.

3D Printing and Thingiverse Login Fix

I recently got a new 3D printer (Ender 3 Pro), and thought I would put up some of the small things I have recently printed. In trying to print things from Thingiverse, I couldn’t login even after making an account. I would get a spinning “Logging in” and it would never end. After looking at the network log, I saw it trying to reach out to https://accounts.thingiverse.com/unverified?username=danberk If you run into this issue, go to that URL with your username and it will send you an email to verify your account. Then the site will allow you to login.

Ruckus ICX 7150 sideways shelf mount

I have been using Ruckus ICX 7150-12P switches at home recently, I wanted to have it more out of the way; so I designed and printed a mount that would mount the switch to the side. It came out well and looks good! I also printed a network cable comb to hold all the cables nicely together.

SD card and USB holder!

This is a nice little SD card, micro SD, and USB holder. I 3M stuck it to the shelf I have, next to the printer.

Mac SE with battery mount

I have already posted about the Mac SE Battery mount I made. I put the design up if anyone is interested.

Email Alerts on Different Platforms

Different network gear I have has had many problems trying to get email alerts working. I thought I would document them. All of these systems use a service gmail address I made on free/public gmail to send alerts to me.

Sophos, and LibreNMS gave me no problems; if you have issues with them drop a comment below and I can post my settings.

Ruckus AP

The trick to getting Ruckus Unleashed, I used “smtp.gmail.com” and port 587. The issue I ran into is the service email I use to send emails had a long password. Ruckus Unleashed v200.8 supports a maximum of 32 character passwords. I would also mention it dumps the password raw into the logs, so make an account you dont care much about.

Unifi Controller

After digging through logs and getting lots of “There was an error sending the test email to x@gmail.com. Failed to send email for unknown reasons.”, I found one post that mentioned a fix for the console log of “fail to send email: api.err.SmtpSendFailed”. You need to once again use smtp.gmail.com, and port 587, but since its TLS, you need to counter intuitively UNCHECK “Enable SSL”.