Projects

Bitbucket: Convert From Standalone ElasticSearch to Embedded OpenSearch

At work I maintain random stacks of software, and sometimes help people with other stacks that they maintain. Recently I was asked to help bring a Atlassian Bitbucket stack up to date. In the past Atlassian always included a built-in ElasticSearch (ES) server. This was used to index code in Bitbucket and allow searching. It’s not a hard requirement for the server to function, but important for user experience.

When an environment moves from Bitbucket Server to Bitbucket enterprise you are supposed to go to a standalone ES over the embedded one for performance. I don’t know if people elsewhere commonly do this, but the stacks I have seen have just continued to use the embedded version. Admittedly, these are smaller instances; at scale I would understand that. That was until recently, when due to a licensing change Atlassian could no longer embed a up to date ElasticSearch. For a while they decided the best way to move forward was to keep bundling the one from before the licensing change (I think 7.10).

This works until you have an infosec team use Nessus and find you have an out-of-date ES sitting around when 7.16, or the 8.0 branch are out. From all that, this one stack had moved to a standalone ES cluster. We also now had to install the Atlassian security plugin into ES; this was not a simple task, and this plugin only supports a few versions of ES, none of which were current. At least then we are at a BETTER spot with security.

Now fast forward a few months of this mess going on, and Atlassian moved Bitbucket from ElasticSearch to OpenSearch. OpenSearch is a fork of ElasticSearch at version 7.10.2 from Amazon to get around these new licensing terms. Normally if you were still using the embedded version of ES, when you did your next upgrade of Bitbucket it would move you to OpenSearch. Because this stack had already moved to standalone instance it did not migrate over. We are now in the worst of both worlds, off the supported path, and can’t get back on it. If you search the Atlassian documentation there are guides on how to move to a standalone version, but not back. A big catch I found was they use default passwords in the embedded version, that are not easy to find, which lead you making it hard to migrate back.

Migrating Back

Below are some notes I have on migrating back. Hopefully they help someone.

There are two main folders we will work in, one is your Atlassian Bitbucket installation folder for this version, I will call it %atlassian-install%, then there is your Bitbucket data folder that moves between your versions, with your upgrades, we will call that %bitbucket-home%. (Note: I did all this on Linux, but I am calling the variables that because it is easy)

Default %atlassian-install% is /opt/atlassian/bitbucket/7.21.7, or your current version. Default %bitbucket-home% is /var/atlassian/application-data/bitbucket, but I tend to move that to /opt.

Under %atlassian-install%/opensearch/plugins/opensearch-security/securityconfig/internal_user.yml is the details Bitbucket needs to connect to this OpenSearch instance. The default password is “bitbucket-changeit”. To create a new hash of a password, the following file needs to be given execute privileges and does not come with that on Linux; %atlassian-install%/opensearch/plugins/opensearch-security/tools/hash.sh .

Go into %bitbucket-home%/shared/bitbucket.properties if you have one, this file is created as you migrate between versions or databases; and remove any legacy elasticsearch username/password/url settings. For example: plugin.search.elasticsearch.baseurl or plugin.search.config.baseurl as shown in the documentation. The properties file overrides settings you have in the instance/database. You may have a SystemD service file to automatically start Bitbucket, this file has the start-bitbucket.sh file starting with -ns or --no-search to run a standalone instance, remove the no search option.

Now start Bitbucket and go to Administration -> Troubleshooting and support tools -> System Information, you will see Search failed to connect. Go to Administration -> Server settings, then enter your new search information there. If you just removed ElasticSearch, and started OpenSearch with the server, all you have to do is make sure the port is right, by default 7992 I believe, then make sure the username is “bitbucket” and the password is “bitbucket-changeit”. If you get a connection error it may be that you have to setup a TLS trust between Bitbucket and Opensearch, but that is outside the scope of this guide.

Below is the default %bitbucket-home%/shared/search/config/opensearch.yml

cluster.name: bitbucket_search
node:
  name: bitbucket_bundled

network.host: _local_
discovery.type: single-node

path:
  logs: ${BITBUCKET_HOME}/log/search
  data: ${BITBUCKET_HOME}/shared/search/data

action.auto_create_index: false

http.port: 7992
transport.tcp.port: 7993

# The OpenSearch security plugin stores its configuration in an index in the cluster itself. On startup if the
# security index doesn't exist yet, sitting this to true will cause the security plugin to read the yml files and
# configure the index using the contents of the files.
plugins.security.allow_default_init_securityindex: true

# Using the yml files with default initialisation, we create a bitbucket user and give it the all_access in-built role.
# However, access to the REST API is disabled by default even for the all_access role so we need to explicitly give
# it permission here so that the bitbucket user can access the OpenSearch REST API.
plugins.security.restapi.roles_enabled: ["all_access"]

# Mandatory TLS setup for transport layer
plugins.security.authcz.admin_dn:
  - CN=BITBUCKET
plugins.security.ssl.transport.enforce_hostname_verification: false
plugins.security.ssl.transport.pemcert_filepath: bitbucket.pem
plugins.security.ssl.transport.pemkey_filepath: bitbucket-key.pem
plugins.security.ssl.transport.pemtrustedcas_filepath: root-ca.pem

# Logs audit events to bitbucket_search_server.json
plugins.security.audit.type: log4j
plugins.security.audit.config.log4j.logger_name: audit
plugins.security.audit.config.log4j.level: INFO
Optiplex 5050 Back view

Dell Optiplex 5050 Micro Windows Server Installation

Recently I was able to pick up some Dell Optiplex 5050 Micros for $60 on eBay. These are tiny machines, with an Intel i5-7500T (4 core/4 Thread) CPU, 8GB of ram, and a 256GB SSD. For $60 they needed a power supply, but those are easy to come by. My plan was to replace my aging Intel NUC that is the core domain services for the house (AD, Radius, CA) and perhaps the aging firewall, if I can figure out how to get a second NIC into the system, more on that later.

My philosophy when running a standalone network (even with internet access) is to have at least 1 of your Domain Controllers (DCs) be a physical box at all times. An alternative is a dedicated hypervisor with local disks, but anyone who has tried to start a VM manually on VMWare knows how painful it can be without any interface to the system other than the command line. In addition, these days it’s easy to make all the DCs virtual, but if you ever have to cold boot your environment; then you run into not having DNS. Following not having DNS, things like vCenter and vSAN can’t come up cleanly, and there are more and more chain on effects. Having a physical machine allows you to bring DNS and core services up first, then start all other services that rely on your domain.

The first task I had was to get one of the Optiplex 5050s ready for Windows Server. I started with upgrading the ram to 16GB, because I had it laying around. After that, since this is an eBay purchase, I updated the firmware/BIOS and ran diagnostics before it touched the home network. The seller was nice enough to install Windows 10 Pro on the machine, which has a license in the BIOS; but I formatted the drive before starting that instance. People are generally nice, but who knows what was in that image. After getting Windows Server 2022 installed I hit my first issue. Server 2022 does not have a driver for the Intel i219-V that is in this chassis.

I tried getting the drivers from the Dell site, but Windows refused to use them because they were for Windows 10, and not Server edition. My current fix for this was going to select the driver, telling it to “Browse my computer for drivers”, letting me pick, then manually selecting the “Intel” “Intel(R) Ethernet Connection (2) I219-V” driver. I had a USB ethernet dongle that worked for me to get online and at least be able to see that driver. Now the box is happily online. The main issue with this technique is that I keep getting an “Optional” Windows Update for an updated driver that seems to never install. I think that is because I installed the Dell driver, but it never runs correctly.

Another thing I try to do with most systems, especially the systems in charge of security is get Virtualization Based Security running. This is a newer Windows feature, where core elements that need to maintain secrets are run in tiny Hyper-V containers. The user never sees it, but this gives added protection to the system. If you run “msinfo32”, you can get an output of its status. Most of the time, you need to enable chipset virtualization support; then add the system feature of “Host Guardian Hyper-V Support”. On older systems (Windows Server 2019) and desktops, I think it’s just called “Hyper-V”, then you get these features enabled.

On paper this machine is 78% faster than the Intel i5-3427U, and that makes a world of difference. The old system took a while to boot, and a while to backup, which is what spurred me to upgrade. This system feels amazingly fast for a $60 system. Keep in mind that it cost less than the Raspberry Pi 4, has Intel, and didn’t have to wait the years Raspberry Pis take right now!

I have the main DC run domain services, DNS, Network Policy Service (RADIUS), and certificate services. For the first two, I just had to install Domain Services and DNS and the system started acting in that role. For NPS I exported the config from the old DC, and then installed the service and imported onto the new one. As a reminder, Domain Services has to be installed first, or if you have NPS/Certificate Services installed, then try to do Domain Services, it will tell you it can’t install. Certificate Services, I added a new CA, stopped the old one’s service, and removed it as an enrollment agent in ADSI. My 802.1x and other certs given out by GPO are short lived, around 90 days; I will wait for the old ones to expire and systems to naturally get newer certs.

The second system I got; I thought I would try to do some hardware hacking. My hope was to repurpose it as a firewall for my aging Dell Optiplex 990 from 2011. To do this I would want to add at least 1 more NIC to the system. I ordered a 1gb ethernet NIC that goes where the WLAN chip goes. At first it did not show up in Linux and I was worried. Turns out the system bios had “wlan” disabled, and by enabling that, it turned on that PCIe channel. Then the card would show up. Having mounted the ethernet port in the extra serial blank this system has did make it look very clean and easy. I had to tuck the wire away as it came from the front of the unit to the back and had the sata drive siting on it. After playing with it a good amount, removing the card, reseating, putting electrical tape under it, I was able to get the line up, but not reliably at 1gb/s, it tended to go down to 100mb/s a lot in coming up. While things like loosening the screw holding it down, and putting electrical tape under it helped, the system was not reliable enough for me to feel comfortable using it for homelab-production. I shaved down the connectors at the end of the card, with them being that large, the screw couldn’t easily get between them. That did not help that much.

In the end I am enjoying the one system as a new DC. And eventually will figure out what I want to do with the other one. With having a NVMe slot, and SATA internally, in addition to being able to go up to 32GB of ram on a low power budget they are very capable little machines.

Stable Diffusion Image, "computer with redhat logo on screen, in a field with mountains and a dinosaur in the background"

Clean Tenable Nessus Scans for RHEL 7 with Podman

There can be an alert misfire for Tenable Nessus plugins 137561, 138032, 142002 based on your YUM repo configuration. This leads to 3 medium alerts that should not be there.

Plugins:

  • RHEL 7 : OpenShift Container Platform 4.3.25 containernetworking-plugins (RHSA-2020:2443) (Plugin 137561)
  • RHEL 7 : OpenShift Container Platform 4.2.36 containernetworking-plugins (RHSA-2020:2592) (Plugin 138032)
  • RHEL 7 / 8 : OpenShift Container Platform 4.6.1 package (RHSA-2020:4297) (Plugin: 142002)

If you have a stack that is using podman with RHEL 7 and does not have the default redhat.repo file, then packages are installed that have newer versions in the OpenShift repos. Normally this would be fine, but the Nessus scanner is supposed to check if you have OpenShift repos enabled, and if not then stop and say the latest versions from RHEL 7 OS is good; but the check fails if you are missing the RHEL 7 OS repos. The OS repo HAS to be enabled also, or the check will show as failing. This situation can easily happen if you have an air gapped system or a system on Satellite where you are not using the default repo in redhat.repo. Luckily the baseurl does not matter, as long as you set the name to “rhel-7-server-rpms”, and I put the name= line in there for good measure, then the check will come back clean.

/etc/yum.repos.d/redhat.repo

[rhel-7-server-rpms]
name = Red Hat Enterprise Linux 7 Server (RPMs)
baseurl=file:///opt/rhel_7_x86_64_os/
enabled = 1
gpgcheck = 0
gpgkey = file:///etc/pki/rpm-gpg/RPM-GPG-KEY-redhat-release

Before (or after) setting that, you will need to disable the YUM Redhat Subscription Manager plugin, or the next time you run “yum” it will wipe your redhat.repo and reload it from subscription manager. To do this, go to /etc/yum/pluginconf.d/subscription-manager.conf and set “enabled=0”. Also # subscription-manager config --rhsm.manage_repos=0

Below are examples of the errors you can see from Nessus.

Remote package installed : containernetworking-plugins-0.8.3-3.el7_8
Should be                : containernetworking-plugins-0.8.6-1.rhaos4.2.el7
OR
Remote package installed : runc-1.0.0-69.rc10.el7_9
Should be : runc-1.0.0-81.rhaos4.6.git5b757d4.el7

The default thinking may be, it says I need to update to the OpenShift packages; then it makes sense to install the OpenShift repos. And if you go get a Redhat developer account to debug this, you have the OpenShift repos there. That is because the developer account gives you a lot of entitlements including OpenShift, and if you add the OpenShift repos to a bunch of systems, you may be liable to get OpenShift licenses, or get errors because those systems do not have the entitlements. The key is the packages say “.el7_8″/”.el7_9″ instead of “.rhaos4.2”. This is a plugin misclassification, not a need for updates.

Note: The image is a random AI generated one: Stable Diffusion Image, “computer with redhat logo on screen, in a field with mountains and a dinosaur in the background”. I think they are fun.

THE µKENBAK-1

Back again with another retro computer kit from the same creator as THE ALTAIR-DUINO, a small quick kit in the µKENBAK-1. The µKENBAK-1 with the µ in front denotes one of the earlier versions of the kit. This is smaller than the original computer kit, compared to the full-sized replica or nanoKENBAK-1 now offered by the creator. This is a small, and simple kit. Running off an Atmel processor (same as Arduino), this little recreation offers a fun, simple front panel, and relatively quick assembly.

Compared to some of the other kits that have been posted here, this one is straight forward to put together. While you have the classic soldering, the kit is all through hole components and is a pleasant hour or so to put together. The most time for me in putting the kit together actually came down to getting the PCB with the stand-offs in the case and lined up with the back holes. This proved to be a difficult, and time-consuming process. You need to pre tighten them on the front panel, which then slides into the case, and line them up with the back holes. Between the standoffs being plastic and wanting to strip, and them wanting to wiggle all over, most of my time went into this instead of soldering. In the end, I got 5/6 in place and called it a day.

Evil Stand-Offs

The creator of this kit shows his experience in creating these kits, in little details, which make the kit a nice experience; one example is the usb extension cable which gives you an easy connection out the back is the perfect length to do the job but not be in your way. Another is the instruction booklet coming with a bunch of examples on how to use the computer, right after the assembly instructions. These instructions come in a nice spiral book included in the box.

The creators website, https://adwaterandstir.com/kenbak/ also goes into detail about the creator of this machine, (the original one in the 1970’s), and its history.

This is one of the easier kits I have done, but enjoyable in its ease to put together. I would recommend this kit to someone who is looking to get started with these kits.

Missing Email Alerts from LibreNMS

I realized recently that I haven’t gotten any alerts from LibreNMS recently, including when I rebooted devices for patching. After going to the “Alert Transport”, and attempting to send a message I got “SNMP Error: Could not authenticate.” Others seem to recently get this as well. (Link)

Turns out after May 31st (although for me it seems more like June 6th, 2022) Google disabled simple password logins for Gmail accounts. You need to enable two factor auth, then enable an app specific password for LibreNMS. This was a good quick guide on how to do that. With LibreNMS sending alerts when something is wrong, but not having a alert that it is working, it may be worth going and checking if you use LibreNMS and Gmail.

Computer Vision for Datacenter Auditing

I am going to start a series of posts of random ideas I have had but not had time to fully implement. The first in this series is a idea I have worked on about ~3 years ago (November 2019)  for being able to audit a datacenter as well as map systems physicals location to their logical one in a network.

The core of the idea is to use cameras in a datacenter to see servers in the rack, these could be security cameras, then use that data to map out the datacenter and save the administrators time from having to manually preform these actions. The process begins by training a machine vision learning model on what a server looks like. Most of the time at work I am working with Dell servers so I thought that was a good starting point. To make the model generic enough I was just attempting to train it on what a 1U server looks like vs what a 2U server looks like.

At this point I needed A LOT of photos of servers with different lighting and angles. I took a bunch myself as a seed set of different racks I had, then I turned to the web. Where could I get a large assortment of photos of Dell servers in different configurations and lightings? The homelab section on reddit! People all the time post their setups at home and what they have. I went through and downloaded several hundreds photos of different peoples setups. Another place to get photos was from eBay where a lot of sellers put up photos of servers in different settings; the downside is that a lot of people reuse the same photos again and again. I don’t know if the internet has yet to figure out what the copyright rules of using photos from online to train a model.

I researched a bunch of different techniques and was playing around with OpenCV, but then found a tutorial that seemed to be in line with what I was looking to do. (This one is also good, and very similar material) I researched different image processing models, and played around with several.

Now that I had the photos, I downloaded GitHub – tzutalin/labelImg: 🖍️ LabelImg is a graphical image annotation tool and label object bounding boxes in images. This is a tool where you go to each photo, select the item you are trying to learn, and label it. This took a while, its fully manual work. A lot of the photos from the web had multiple servers in the photo, and each one would need to be selected. This proved to be one of the more time consuming parts of the project. I had to manipulate the photos to allow the rectangular bounding boxes to be able to fit the servers, even when the photos are at weird angles.

I had to pick some of the photos to be the training set, than other photos to be the testing set. With that metadata ready and everything marked, I converted the final metadata from XML to CSV, using xml_to_csv.py provided at the above example repos. That was then fed into Tensorflow. The system I had to start with for this other than a laptop was a CentOS 7 server, this proved to be very annoying because some dependencies such as protobuf were not available at new enough versions and had to be custom compiled.

It was time to let the model run for a while and see what it could learn. Several important things were learnt in this process. First, if you have GPUs makes sure you have a Tensorflow that is compiled and ready to use them. The speed you speed you get with and without them is kind of crazy. Also, more RAM and GPU helps a lot speed up the process. At first I was playing with this on just a laptop, and that one didn’t have the GPU drivers for CUDA. This was taking DAYS to work on the model. Later I switched to using GPUs I had in a server, and this greatly increased the iteration cycle speed.

Off the bat it was able to get a decent percentage recognition of the servers in the photos I had presented it! I do think a lot of the photos I then tested it on were fairly ideal conditions, with good lighting and camera angle. This may give a better than real world experience with it working. To improve the model I can always find more photos and train it with more images. I was able to get the model to recognize about 80% of the servers in racks I showed it at this time. Another factor that could help in the future is the evolution of cameras. A lot of places are replacing 720p/1080P cameras with 4k cameras, the more resolution the system has to work with the better.

The next step I wanted to do was start matching physical location to logical. The idea behind this is, I can find regions in a photo or video where servers are, and each server through its iDrac/IPMI allows me to blink front chassis lights. So one host at a time I will have automation send the command to blink the front chassis lights, and perhaps some lights on the HDDs, then scan for which region in the photo has started to blink!

This is the idea I have slowly worked on for the last little while, I have prototypes of most of it working, but have not had a lot of time to put into it.  The hope would be we could use existing cameras to get the footage we need to map existing datacenters we have. Then perhaps in the future port this system to something like Hololens, or Apple/Meta AR system. Once we have that mapping, now we can start to draw out the physical servers and their location in the world/racks on a webpage, and make it easier for people working in a datacenter to find boxes they need. Hopefully one day allowing for people to click a server on a webpage, and then connect into its controller without a human painstakingly going to each box and doing this mapping. Of course all of this is fixed by a team labeling each server, but where is the fun in that.

Cisco ISE 2.X Certificate Expiration

Quick post: I had a HA pair of ISE boxes in a lab the other day have the certificates that I made with a Windows Certificate Authority expire the other day and I ran into some odd behavior. To be clear, in this scenario, the certificates had a valid chain of trust, but it was past its expiration date.

I logged in after realizing this and had odd behavior, node-A could not read node-Bs certificates. Both nodes said they were no longer on domain, even though the domain disagreed and I logged in with domain credentials that were recently changed. Then when I went to make a Certificate Signing Request (CSR), I was able to make it, but when I went to download it I got a generic message of “Cannot connect to node-a”. At the same time all these issues were going on, under “Node Status” on the dashboard, both nodes were sharing health data.

In the end, ISE gets weird when the cert date has expired. I generated a new self signed cert for node-A. Then deleted the expired certs because the system didnt want me to make a CSR for the same thing it thought it had a cert for already. This allowed me to then properly make a CSR and export it. That gave me “ciscoisenodea.pem”, I brought that over to my setup Windows CA, and with a admin command prompt ran certreq -submit -attrib "CertificateTemplate:WebServer" ciscoisenodea.pem . Saved that to my local desktop, and went into ISE to Bind it to the CSR. Node-A then rebooted. All of a sudden things like the domain pairing, started showing they were working again. Then the second node, I did the same process, and all of a sudden everything was happy again. Note: make sure you have a your admin backup password, one of the nodes DID refuse to talk to AD and I had to use that, while the other one said it wasn’t on the domain, but did work…

Hope this helps someone out there!

Ruckus ICX 7150-C12P Switch Repair

A while ago I purchased a Ruckus ICX 7150-c12p off eBay to use at home. It gives 14x1gb/s ports, and 2 SFP+ ports. The SFP+ ports are limited to 1gb/s by default, and there is a honor system license for upgrading them to 10gb/s. These switches go for $600 – $1200 depending on where you get them and which license you get with it (1gb/s vs 10gb/s). The switch is also POE, and can do 4 POE+ (30 watt) ports. I had one of these switches and it worked great. I wanted to get a second one to replace the WiFi link I was using across my apartment with a fiber link.

Instead of paying ~$250, which was their going rate on eBay; I saw a forum post about replacing this models power supply, and thought I would give that a shot. I got a broken switch for $45, and then a PSU for $50. The PSU I used was a SL Power LB130S56K 56V 2.32 130W. Armed with someone’s photos of doing this repair it ended up going fine. The hardest part of the whole operation is that the pins going onto the main board are reversed from what the power supply comes with, so you need to flip them. I have been running the unit for almost 2 years now without issue.

This model of switch is great because of its features and is fanless. The fanless-ness part of it is nice for homelabs near your desk, because the switches are silent. Because they are fanless, they cant have anything put on top of them, and need some room to breath. I think a lot of the ones you see online dead are because someone didn’t give it enough air, and the PSU died. Note when looking for a similar dead switch on eBay, you really want the seller saying “when plugged in nothing happens”, not “it periodically blinks” because that could be bad ram and its in a boot loop.

Having run two of these switches for over a year, I can give some feedback. I really like them. I have the two I have in a stack, I login once and manage both. When it comes time for firmware updates you SCP the file to the management IP, and it downloads the file to both, and then flashes and reloads. I came from using Cisco gear usually, or sometimes Arista; the CLI is a bit different, and Ruckus handles VLAN setup a bit weird, but once you get used to it, it makes sense. They are solid switches, with POE, that you can set and forget for a while.

New PC Build 2022

Having built my PC a few years ago, I was thinking about upgrading it, but with chip prices being what they are, and graphics cards costing more than a new car, I thought I would wait. Then a friend of mine happen to have an AMD 5800X that he was willing to give me a deal on… After years of having the custom case, while it was nice, it was HUGE and made it difficult to setup a desk in a tiny New York apartment. I used this opportunity to shrink a bit, and update some of the components. I say some, because some of them (like the graphics card) were going to stay to save cost.

Old case being taken apart, its a bit of a mess

At this point, this post is mostly a standard PC build, with a few hiccups along the way. Looking on the Small Form Factor PC subreddit, and https://caseend.com/ (a website for small pc cases), I settled on the ZZAW C3. It is relatively small, supports Micro-ATX, full size graphics cards, and all-in-one water coolers in a ~22 liter case. I always try to get Micro-ATX over Mini-ITX for my desktop because you get more PCI slots; as well as 4 DIMMs for RAM instead of 2. I also wanted to try one of these all-in-one water coolers, since I never had and thought I could get good results (and a quiet case).

The case came nicely wrapped. There were not a lot of instructions on how to put the case and system together, you kind of just had to know. That took a bit of trial and error. There are a few screws that are very similar sizes, and not labeled. The whole setup went smoothly after that except that getting the cooler to fit in the case was a bit of a challenge, and the motherboard… So, the motherboard… I got a AsRock X570 Pro4 motherboard, it had all the features I wanted. I get the case put together, I installed my previous power supply, I kept the RAM and graphics card, moved my SSD over (I had a PCIe Gen 4 Samsung 980 Pro on order, but it hadn’t come in yet), put thermal paste on the CPU, install the cooler AND… nothing. System will not boot.

There are lights on the motherboard saying CPU, RAM, and DISK failed. I start searching online and trying different things. A bit in, someone says “make sure your motherboard is updated to support 5000 series AMD”. I just got this board, its fairly new, it has to work right? Well turns out you need BIOS version 3.20 to support the CPU I have, AsRock is at 4.20 for this motherboard. After taking my old 2600X out of my old motherboard, taking out the new CPU, cleaning the thermal paste, reseating, re-thermal pasting the 2600X, turns out the board shipped with version 3.10… One revision before what I needed. I updated the BIOS, then swapped the CPU back, doing all those steps again. This time, the system worked.

I later got the Samsung 980 Pro SSD, which was a tiny bit of a headache to migrate over because I had Bitlocker on and trying to relocate the bootloader was not straight forward. I kept getting “Boot Device Not Found” and eventually somehow doing a system restore to before the last Windows Update got Windows 10 to reinstall the bootloader on the drive. I have no idea why that would have fixed it. I had disabled Bitlocker before starting this ordeal, but that just does it one reboot, I really needed to just fully disable it to save me typing in the recovery pin many times.

The system is working well. The only remaining issue, that is very odd, but I am just living with is whenever the system attempts to do a restart, it freezes before coming back. It properly shuts down, and if I hold the power button and then turn it back on everything is fine. But it refuses to gracefully restart. It also will fully shutdown fine. Odd…

Towing a U-Haul with a Subaru Forester

Recently I was helping a family member move states. They had some larger, but light, furniture to move and we were trying to figure out a solution. Having recently gotten a 2021 Subaru Forester with a tow hitch on it I thought I would help them move those items with a U-Haul trailer. I could not find a lot online about this, other than a few Reddit/Forum posts; so I thought I would post about what I learned.

One of the main reasons we went with renting a U-Haul we would tow over a truck is cost. U-Haul trucks cost is based on distance, U-Haul trailers come with unlimited miles. The estimate we got for renting a truck was around $1,300 for 3 days, the trailer was $550. We were renting for 4 days.

First the 2021 Forester (non-wilderness package) is rated to tow 1,500lbs in the US. That is the big issue and ceiling that you will hit. The 4×8 is 850lbs empty, and the 5×8 is 900lbs empty. That leaves us with only 600lbs of capacity when getting the larger one that can fit a bed. Is this a hard ceiling? No, but as people on Reddit and other sites have pointed out it can wear out your car (mostly the transmission) more if you tow over that, specially if you do this often. I knew the route I was going to take, and that 98% of it would be flat interstates. That, along with knowing I was towing lighter things, made me less worried about the weight.

The next thing to worry about is how to hook it up. The U-Hauls come with a 2 inch ball mount coupler. My Forester had the hitch installed from Subaru. Subaru puts in a Class I, 1-1/4 inch receiver. I used the following ball mount, which ended up working perfectly.

Ball Mount: https://www.amazon.com/CURT-45572-Euro-Mount-Ball/dp/B003B3GX5E/

I spoke to U-Haul on the phone and they said you want the hitch to be level at about 18.5 to 18.75 inches off the ground. The Forester mount is about 14 inches off the ground, and the above posted ball mount adds 4 5/8s inches, giving you the height you want.

I got the CURT 21410 Trailer hitch pin Amazon recommended. DO NOT GET THIS ONE. It is 2 inches long, with the extra metal around the mount for the hitch, this pin would not fit the stock Subaru hitch. It didn’t fit just by a hair. I ended up at the last minute going to Home Depot and getting a 2 1/4 inch long, 1/2 inch thick pin that fit.

Now that we had the hitch, we are off! I ordered a 5×8 U-Haul 2 months before we needed it on U-Hauls website, I quickly got a call saying they don’t have one at the local rental place, but would get back to me within 48 hours of the rental to tell me where to pick it up. The call never came. With less than 24 hours left, I call the main 1-800 number, and work with a nice lady on the phone to find where I can pick one up. There was a different facility than the only I selected online, 30 miles away that had it available.

The next day I was there when I would them I would be to pick it up, and no one was there. There was a sign on the door saying “for help call X number”, and then someone answered and within 5 minutes was over to help me. The pickup place was a small business attached to a self storage place who check once a day or so for pickups, and since at the last minute I changed it, it wasn’t noticed. I was just happy the person came so quick to help me get going.

With having the hitch already on the car, pickup took less than 5 minutes. I didn’t know if they would inspect the car, or check anything, but he said just back up to the trailer and we will hook it up. I asked the older gentleman if there is any advice he would have for someone who has not towed before. He said to leave extra stopping room, and make wide turns. The trailer dropped right on the ball, then we attached the wiring for the lights, and checked they were working. The standard 4 pin hookup the Forester hitch came with was exactly what the trailer had. Another important thing I was told for towing, CROSS THE SAFETY CHAINS BEFORE ATTACHING TO THE CAR. Apparently if you don’t cross the chains you can get a ticket, and State Troopers LOVE to give out tickets for it.

The smaller trailers (4×8, 5×8) do not have breaks, they fully rely on your cars brakes. This is something to think about; for instance, parking on a hill, all the weight of the trailer rests on your car and its transmission. I used the parking break a lot when parking the trailer. When I grabbed the trailer it had 2 wood blocks under the tires to keep it in place, I asked to take those with me (which ended up being a very good idea). They helped when parking in some locations, as well as when you want to take the trailer off, if you don’t have them it will want to roll because it has no brakes of its own. When I went to drop it off, the other U-Haul location (I did a one way trip), required those blocks to hold it the trailer in place, so I once again was glad I had them. They may have had some more of their own, but get the blocks when you pick it up, you’ll be glad you did.

The trailer says max speed 55, and after a bit of getting used to it, I felt comfortable with that. You just have to get used to being in the right lane, and giving plenty of room when changing lanes. It takes a while to start and stop, be prepared for that. Some people online mentioned and I felt one or two times, if you brake too hard, and then the trailer pushes forward on your car, the automatic transmission does not like that ad can rev up in situations you wouldn’t want it to.

I hope this post helps anyone who has a similar situation, and feel free to drop questions or your experience!