FortiGate 80F to Unifi Security Gateway Pro 4 IPSec Tunnel Issues

I have recently replaced an older Cisco ASA 5550 with a FortiGate 80F. Firewalls are not exactly in my guru status, so I do bring in a networking consultant company to help with these types of things. They are actually the ones the recommended the FortiGate 80F. However, they are quite busy, so it’s up to me to learn the systems and troubleshoot much of it, especially when it falls out of the FortiGate brand.

For years, the Cisco ASA 5550 had an IPSec Tunnel that worked flawlessly with the Unifi Security Gateway Pro 4 (USG-PRO-4). But the Cisco did not work well for more modern firewalls with other companies and AWS, which is one of the primary reasons I changed it out.

Things seemed to go well at first after a weekend install, though we didn’t do much monitoring of the specific tunnel as we didn’t get any complaints, and it’s not used that often under sustained traffic. However, the next weekend I received a complaint that RDP from the FortiGate SSL-VPN tunnel, through the USG tunnel, was failing about every 1.5 minutes.

Being new to the FortiGate, and not having touch the Unifi interface in years, it was cold turkey learning for me.

The network consultants and I went through a great deal of troubleshooting, including turning off DTLS on SSL-VPN. They saw packet errors, via the counters, on the tunnel using the following command on the FortiGate:

get vpn ipsec tunnel summary

which returned

tx packets: 1992 bytes: 1092273 errors: 134

The USG side is connected to the Internet via a cable modem from Cox on a business plan. It isn’t exactly fiber and can be prone to oversubscription (as we saw during the pandemic), however, the Internet connection wasn’t showing any symptoms there, and we didn’t show much if any, packet loss or errors via some long ping cycles to services such as Google DNS ( and the public side of the FortiGate. They saw about 300ms ping times with some Jitter coming from their network, but we didn’t feel that this really was unordinary for a cable modem. We even opened a Tier 3 ticket with Cox and they monitored the traffic for 24 hours, seeing no issues.

We did try some diagnostic tools such as iperf3 and WinMTR, but those didn’t really give us anything useful.

Dead Peer Detection (DPD) was disabled on the IPsec tunnel. DTLS was disabled on the SSL-VPN. And Perfect Forward Secrecy (PFS) was disabled.

On the FortiGate, I saw a lot of “phase 1 negotiate error w/ PAYLOAD-MALFORMED” and “phase 2 negotiate error w/ progress IPsec phase 2” errors in the VPN events section of the FortiNet UI.

After off DPD and PFS, I realized that had a big impact on the frequency of tunnel reconnections, so I did some more digging.

After disabling PFS, the “PAYLOAD-MALFORMED” errors went away and I started seeing “INVALID-ID-INFORMATION”.

I did some playing around with using IKEv2 on Phase 1. But that did not work and reverted back to IKEv1. I also messed with the AES and SH1 encryption and hashing along with the DH groups. When I change these, the tunnel starts to work in many cases. But then the issues just come back after awhile. For now I settled on AES128-SHA1 with DH 5.

I disabled allowing the USG VPN to access the site-to-site tunnel. This was causing error noise as the USG VPN IP Pool was not setup to be received on the FortiGate. I also corrected/narrowed the Phase 1 and Phase 2 selector encryption and hashing options on the FortiGate. The FortiGate had copied over settings from the Cisco 5550, which was fairly broad on allowing many options, But here, those were just causing error noise and negotiation issues. These are common across the USG for phase 1 and phase 2, but separate, hidden away under advanced, in the FortiGate. After doing this the packet errors went away!

At this time I learned how to output IPSec logs on the FortiGate via SSH:

diagnose debug duration 480
diagnose debug console timestamp enable
diagnose vpn ike log-filter dst-addr4
dia debug application ike -1
dia vpn ike gateway flush name %Tunnel-Name%
dia vpn tunnel stat flush %Tunnel-Name%
dia deb en
diagnose debug disable
diagnose debug reset

I started with -255 verbosity, which only gave me this:

ike 0:site1:3903: nat unavailable
ike 0:site1:3903:Site1-sslvpn:578090: quick-mode negotiation failed due to retry timeout
ike 0:site1:3903:Site1-sslvpn:578119: quick-mode negotiation failed due to retry timeout
ike 0:site1:3903:Site1-sslvpn:578154: quick-mode negotiation failed due to retry timeout

But once I started using -1, it gave me a great deal more useful information. This started giving me “phase 2 proposals not being received”, which was a useful clue.

This round it started failing at Phase 2 with this error around 60 hours in.

I then started doing some digging on the USG to see what I can find via SSH. VPN logs are stored on the USG at /var/log/charon.log.

tail -f /var/log/charon.log

However, these did not prove to be much use. I then found a real-time console output of the IPsec connection information using “swanctl”. See

sudo swanctl --log

at which point I found these, when the cycle of constant tunnel reconnections occurs:

invalid HASH_V1 payload length, decryption failed?

It turns out the USG-PRO-4 runs the StrongSwan server, version 5.2.2, for IPSec, and the Ubuntu version is from 2019. From what I’m reading, this is the same server the Cisco ASA ran. I would assume around the same version. This version is from 2015, and there have been many gripes with issues in this version area.

After some investigation, I found that people were resolving this with a shorter pre-shared key and removing special characters. This was an IPSec connection from a mobile device. Says Android 7 worked fine, but started getting this with Android 9. Though, this seemed to appear in earlier minor versions. So I’m not confident this is related.

I’m also seeing suggestions to dumb down the encryption even further from AES128-SHA1 to 3DES-SHA1.

Now granted, it takes about 4 days for this issue to arise, so it’s possible they didn’t wait long enough and follow up in the forum. But it’s worth a shot to simplify the pre-shared key. Currently, 22 characters consisting of upper/lower and numeric. Going to just 8 upper/lower characters.

Interestingly enough, I found the config files on the USG for Strongswan that contain the IPSec/VPN configs and pre-shared keys.

The pre-shared rekey didn’t work, however, I’m not positive I got it to the state where that mattered. After a number of tries to low-key reset things (config reload, tunnel down/up, etc) it seemed that only ‘sudo ipsec restart’ (restarting the ipsec/strongswan service on the USG took care of it. A provision to the USG didn’t do the trick, not sure why.

sudo ipsec restart

So if this problem crops up again (in 4 days?), I’ll create a cron job to restart ipsec at 3am each day.

In conclusion, it appears the major issues are:

  • Dated version of the USG’s strongSwan server from 2015, even on the current firmware version
    • Potentially reducing the pre-shared key complexity/size (TBD)
  • Removing encryption/hashing options that do not exist specifically on each IPSec’s configuration
  • Removing the auto-generated tunnel selectors on the USG that do not have a policy on the FortiGate

Here is what you want to see in diagnostic console output on the FortiGate for phase 1:

2023-05-25 17:55:47.927110 ike 0:Site1:5673:873531: peer proposal is: peer:0:, me:0:
2023-05-25 17:55:47.927138 ike 0:Site1:5673:Site1:873531: trying
2023-05-25 17:55:47.927480 ike 0:Site1:5673:Site1:873531: matched phase2
2023-05-25 17:55:47.927509 ike 0:Site1:5673:Site1:873531: autokey
2023-05-25 17:55:47.927551 ike 0:Site1:5673:Site1:873531: my proposal:
2023-05-25 17:55:47.927575 ike 0:Site1:5673:Site1:873531: proposal id = 1:
2023-05-25 17:55:47.927598 ike 0:Site1:5673:Site1:873531:   protocol id = IPSEC_ESP:
2023-05-25 17:55:47.927621 ike 0:Site1:5673:Site1:873531:      trans_id = ESP_AES_CBC (key_len = 128)
2023-05-25 17:55:47.927645 ike 0:Site1:5673:Site1:873531:      encapsulation = ENCAPSULATION_MODE_TUNNEL
2023-05-25 17:55:47.927668 ike 0:Site1:5673:Site1:873531:         type = AUTH_ALG, val=SHA1
2023-05-25 17:55:47.927696 ike 0:Site1:5673:Site1:873531: incoming proposal:
2023-05-25 17:55:47.927718 ike 0:Site1:5673:Site1:873531: proposal id = 0:
2023-05-25 17:55:47.927741 ike 0:Site1:5673:Site1:873531:   protocol id = IPSEC_ESP:
2023-05-25 17:55:47.927763 ike 0:Site1:5673:Site1:873531:      trans_id = ESP_AES_CBC (key_len = 128)
2023-05-25 17:55:47.927786 ike 0:Site1:5673:Site1:873531:      encapsulation = ENCAPSULATION_MODE_TUNNEL
2023-05-25 17:55:47.927813 ike 0:Site1:5673:Site1:873531:         type = AUTH_ALG, val=SHA1
2023-05-25 17:55:47.927844 ike 0:Site1:5673:Site1:873531: negotiation result
2023-05-25 17:55:47.927867 ike 0:Site1:5673:Site1:873531: proposal id = 0:
2023-05-25 17:55:47.927889 ike 0:Site1:5673:Site1:873531:   protocol id = IPSEC_ESP:
2023-05-25 17:55:47.927912 ike 0:Site1:5673:Site1:873531:      trans_id = ESP_AES_CBC (key_len = 128)
2023-05-25 17:55:47.927935 ike 0:Site1:5673:Site1:873531:      encapsulation = ENCAPSULATION_MODE_TUNNEL
2023-05-25 17:55:47.927957 ike 0:Site1:5673:Site1:873531:         type = AUTH_ALG, val=SHA1
2023-05-25 17:55:47.927979 ike 0:Site1:5673:Site1:873531: using tunnel mode.

When phase 2 expires, you want to see this:

2023-05-25 18:03:36.056951 ike 0:Site1: IPsec SA {id}/{id} hard expired 23> SA count 2 of 4
2023-05-25 18:03:36.057106 ike 0:Site1:5673: send IPsec SA delete, spi {id}
2023-05-25 18:03:36.057224 ike 0:Site1:5673: enc {id}
2023-05-25 18:03:36.057273 ike 0:Site1:5673: out {id}
2023-05-25 18:03:36.057361 ike 0:Site1:5673: sent IKE msg (IPsec SA_DELETE-NOTIFY):>, len=76, vrf=0, id={id}/{id}:{id}

If all goes well, I’ll look into adding back in DPD, PFS, and DTLS as well as increasing the encryption, hashing, and DH levels.

Other UniFi USG Commands for IPSec:

sudo ipsec statusall
sudo ipsec up <connection_name>
sudo ipsec down <connection_name>

Now that I’m aware of the legacy server versions on the USG-PRO-4, the end-game is to replace the device, perhaps with a EdgeRouter X. The current firmware is up-to-date and I’ve seen Ubiquiti’s lack of update and support for this item, even though it’s not marked end-of-life.

#80f, #fortigate, #ipsec, #networking, #security-gateway, #tunnel, #unifi, #usg-pro-4

Copying AWS EC2 Tags to EBS Using PowerShell

These days when creating an EC2 instance in the AWS console UI, by default, the tags are duplicated across Elastic Block Service (EBS) and Elastic Network Interface (ENI) attached resources being created. But what happens if you have an older instance that didn’t do that or you add tags to an EC2 instance and need it to reflect across attached resources? It’s easy when you only have a few EC2 instances. Just copy/paste in the console UI. But what if you have them in bulk?

I did my Googling and came across what seemed to be the quickest solution: AWS PowerShell (yes, I’m a Windows user)

I came across “Tag AWS EC2 EBS volumes with the instance name tag” by Alex Neihaus written in 2017 and modified it slightly to fit my needs. But as a new user to AWS PowerShell, it took me a little bit of a learning curve, especially as the method has changed from a legacy Windows-specific, single, large-module version of AWS Tools for PowerShell. Instead, I used a modularized version of AWS Tools for PowerShell where each AWS service is supported by its own individual, small module, with shared support modules AWS.Tools.Common and AWS.Tools.Installer.

Continue reading

Poor Man’s Storage Data Wipe On Windows 10/11

The usual person in the tech industry will inevitably have a storage medium you “must” secure before letting it out of your possession. This could mean something as simple as a personal drive with financial records, personal projects, and embarrassing photos that made their way to Facebook anyway.

First, these opinions are my own, and I am not a data destruction expert. Take this as you will, but do your due diligence for your needs or wants. Do not use my advice if you are representing the government, a financial service, a publicly held company, or a health care organization. There are legal penalties for failure to sanitize data using specific methods for your entity.

See “Tutorial on Disk Drive Data Sanitization” by Gordon Huges, UCSD CMMR and Tom Coughlin for great in-depth information from UC San Diego.

Continue reading

#clean, #hard-drive, #hdd, #sanitize, #ssd, #storage, #wipe

Be Aware of Mura CMS Security Issues

Mura CMS was a popular open-source content management system (CMS) written in ColdFusion. While initially an open-source project maintained by, it was re-licensed as a closed-source cloud application with the release of Mura CMS v10 in 2020. There are forked open-source projects based on the last open-source release of Mura CMS 7.1, including Masa CMS – which is currently maintained at v7.4.

Mura CMS 7.x and Masa CMS versions v7.4.0-beta and earlier contain an authentication bypass vulnerability allowing an unauthenticated attacker to log in as any site member or system user. There is a conditional logic flaw in the “remember me” code. Simply, some well-formed steps can send arguments.userHash variable with no value, creating a true value when looking to see if arguments.userHash is empty or arguments.userHash equals the user’s hash. This is accomplished by passing an empty userHash cookie value if using Lucee, or passing in a single irrelevant character if Adobe ColdFusion.

For this article, I am only focusing on the open-source version. Mura CMS 10 has a different CVE (2022-47003). The Common Vulnerabilities and Exposures (CVE) database defined this vulnerability:

CVE-2022-47002 – Authentication Bypass Vulnerability in Masa CMS (Base Score: 9.8 CRITICAL)
A vulnerability in the Remember Me function of Masa CMS v7.2, 7.3, and 7.4-beta allows attackers to bypass authentication via a crafted web request.
Fixed Version(s): Masa CMS v7.2.5, Masa CMS v7.3.10, Masa v7.4.0-beta.3 and later

Continue reading

#cms, #masa, #mura

Debloat Windows 11

Perform the following at your own risk. Microsoft does not support this program, which is maintained by an individual I do not know personally nor know their reputation well.
It is recommended to perform this cleanup only after a clean install. Using this after customizing or installing programs may end with undesired consequnces, such as missing required programs. As an example, QuickBooks requires XPS for PDF utilities (which can not be fixed with Intuit's fix tool automatically). It is highly recommended to take a snapshot or backup before proceeding.

My job and home life do not require much desktop workstation maintenance. Most CF Webtools staff are competent with their environment or use a client’s pre-packaged workstation. However, I set up workstations here and there for different reasons, mostly Windows 11.

But as you well know, Windows 11 comes with bloatware. I would manually uninstall as many programs and apps as I could. But some are forced to take residence.


Check out “BloatyNosy” (Bloaty ‘n Nosy). Self-described as “streamlines and houses all the essential settings under one app and allows you to disable and remove unnecessary features with just a simple click.”

This app is born as a way to simplify and combine “Privatezilla”, “Bloatbox for Windows 10”, and “ThisIsWin11 for Windows 11”.

After downloading the app from GitHub, you must extract and run the “BloatyNosy.exe” file. Press “Run anyway” if Microsoft Defender SmartScreen prevents you from running an unrecognized app from starting. You may bypass the “Torjan:Script/Wacatac.B!ml” virus warning at your own risk. It appears to be a false positive.

Press the “Analyze Windows x Build x.x” button to scan your computer for recommended actions. You will see results similar to this: (scroll up for more)

If you press the down arrow and press “Fix problems”, it will potentially do some things you do not prefer, such as setting a dark theme and enabling the Linux subsystem.

You may either press this “Fix problems” button and go with their recommendations listed in the list or press the (not so obvious) link below the list to choose what to do and not to do.

After completing your selections, press the “Fix problems” button.

You may have to manually input some confirmations, such as agreeing to source agreement terms when winget runs.

Once the checklist is complete, it will silently sit there without feedback. I recommend opening Task Manager and ensuring the “BlotyNosy” app is at 0% processor, then reboot. Task Manager can be found by right-clicking the Start menu icon and selecting it from the context menu.


You will be left with programs such as “Xbox Game Bar” and “Xbox Live”. You will also notice bloatware apps listed in the start menu you thought would have been removed. However, they appear linked to installers and can be “unpinned “or “uninstalled” individually by right-clicking the icon. These include such programs as WhatsApp, Spotify, Disney+, TikTok, Instagram, and Messenger.

Other Options

You can also use “BloatFinder”, which removes pre-installed apps manually, and “WinModder”, which allows you to apply code snippets based on PowerShell and community scripts, individually. Also, check out “I want to set up Windows 11 for the first time”, where you can call a Windows 11 Setup/Assistant, customize your system step by step, and debloat it. And lastly, InstaPackage is integrated, which allows you to install some of the most important apps with a click and in bulk.

On the WinModder app, you may need to install the mods signature file by pressing the link:

#bloat, #windows-11

Purchasing Adobe ColdFusion

Are you in the market or considering purchasing Adobe ColdFusion’s full or upgrade license?

While you always have the option of going straight to Adobe, I recommend purchasing through a reseller. You will typically see a 5% discount and up to 25% depending on the sale.

Do you need help with installation, upgrades, or troubleshooting? Contact me at CF Webtools to hire ColdFusion experts and get where you are going. I will be happy to discuss your situation and give you advice up-front.


Integral is a great business headquartered in Germany and has regional offices in the UK and the USA. They are the parent company of FusionReactor, and their representative at the ColdFusion conferences is always very helpful and knowledgeable. They have a dedicated ColdFusion software store at:

They typically have about a 5% discount and typically offer a 10% discount on their FusionReactor Application Performance Monitor when purchased together as a 1-year subscription.

I highly recommend a FusionReactor Standard Edition at a minimum for tunning and debugging. Especially useful when troubleshooting performance issues or crashes. Their Enterprise and Ultimate editions are even better while logging data to the cloud.

The Adobe ColdFusion release cycle is about once every 2-3 years. If you are about two years into the release cycle, I recommend purchasing the “Platinum Support” option. Not only will this provide you with 1 year of support, but it will provide you a perceived 50% discount on the next upgrade that is “pre-paid”. Meaning you automatically receive the next major version of ColdFusion. Purchasing within 1-2 years can be a gamble depending upon if you expect to use support. This support package is good only for a 1-year term, however the upgrade key remains available even if you do not upgrade within that support subscription period.

Atypical Upgrade Discounts

FusionReactor will also sometime offer a 25% discount on editions that would be normally full price if you are past the previous year’s installation. Typically you can upgrade from the previous edition for about a 50% discount. However, anything older than the previous edition requires a full purchase. A commitment to subscribe to FusionReactor for one year used to be the requirement, but is currently no longer required.

Q1 2023 Discount Link: (good until the end of Feb 2023)

Coalesce Solutions

Coalesce Solutions is headquartered in Memphis, TN, and is an alternative to Integral. I met them at the last ColdFusion conference, and they were very helpful. We use their AWS Marketplace solutions as needed for AWS EC2 instances. I do not have first-hand experience with their ColdFusion licensing options, but they do have a store available at:

Their full and upgrade pricing is similar to Integral’s.


Learning “AWS Backup” Restorations for On-Prem VMWare VMs

CF Webtools has maintained VMWare ESXi guest OS instances, managed by vCenter, for about 7 years. They are a mix of Linux and Windows Server OSs and are maintained at a secure and redundant co-location data center. While an expensive up-front investment, it has paid for itself over those years, and we have a plan to continue that solution for about another 5 years. A recent upgrade to the next major version proved that virtual machines take a fraction of time for maintenance compared to bare metal instances. Granted, there’s some spin-up time when things work for so long, and you must remember, research, and troubleshoot procedures. Managed cloud takes almost all that time out of the equation, making it my favorite. Though I do miss hands-on hardware here and there.

Some of our on-prem VMs are critical, and some are not. The critical ones have always been backed up with different solutions, depending upon what they are and what the recovery needs look like. However, almost all have come with challenges. So I wanted to look for a VM snapshot-based cloud backup solution that I could trust and would be budget-friendly.

My first direction was to research Veeam. Their solution is very well known. However, it was a struggle to get the attention of Veeam and CDW as a small business without an existing account. I was able to lean on one of our hardware vendors, xByte, who hooked us up with one of their Veeam partners. But it was determined that it was fairly costly with a per-instance license model compared to our existing solutions. So I continued my search.

I then found AWS Backup has an on-prem VMWare solution. AWS Backup is relatively new to the backup game, but its implementations are continually growing. We currently use that service for all our AWS EC2 backups. That service was a “God send” after numerous awful implementations of custom Lambda/CloudWatch scripts and an EBS Automation method. Finally, a solution for what should have been around since the start of EC2.

As of November 2021, AWS Backup offers backup for on-prem VMWare vCenter servers. You must install their Storage Gateway virtual appliance as the “middleman” agent. I was hoping for an “agentless” solution; however, we only pay $0.05/GB-Mo warm storage and $0.01/GB-Mo Cold Storage. That’s a considerable saving, considering we do not have to pay for a license per instance, and there are no incoming bandwidth fees! We will have to pay bandwidth for on-prem restores, but considering that is very rarely done, and bandwidth is relatively cheap, it’s a non-issue. We’d have to pay for storage anyway, so there’s no change.

Another significant advantage is we get a single backup solution for both on-prem and AWS Cloud. It’s one less piece of software we must be familiar with, document, troubleshoot, and keep updated. Outside of an office domain controller, we also anticipate a complete cutover to AWS in 5 years.

Continue reading

#aws-backup, #backup, #vm, #vmware, #vsphere

Wiki.js Active Directory Authentication Configuration

I have recently taken the opportunity to explore Wiki.js as a replacement for a MediaWiki system.

Wiki.js Docs Example

You can add authentication strategies from sources such as Auth0, Azure AD, Facebook, GitHub, Google, LDAP / Active Directory, OAuth2, Slack, and a number more. The current network has a Domain Controller with an Active Directory, so I wanted to integrate that.

The first thing that came to light was that Active Directory (AD) will talk in LDAP on port 389 and LDAPS (secure) on port 636 natively. Sounds great! However, not being an LDAP or Active Directory expert and a lack of documentation from Wiki.js, this became a challenge of “try and sees”.

Continue reading

#active-directory, #ldap, #wiki, #wiki-js

Upgrading PostgreSQL on Docker Compose

I took the dive into Docker Containers on Linux. I’ve been meaning to do this for quite some time, but really my specialty of ColdFusion and Microsoft SQL Server really doesn’t require much in the way of containers.

However, my current project is to convert our internal wiki from WikiPedia to Wiki.js. During this process, I’ve learned about Docker and Docker Compose. I’m currently running Docker 20.10.12 and Docker compose V2. So I use “docker compose” rather than “docker-compose”. This is being run on Amazon Linux 2, on-premise. These steps should work for any RHEL-based Linux distro and beyond.

During this process, I somehow ended up running PostgreSQL 11 when the latest version is 14. I already had data in the database, so I thought it’d just be best to learn how to upgrade PostgreSQL, which is not as simple as changing the version number in the docker-compose.yml config file.

Thank you to José Postiga with Better Programming for getting started with the “How to Upgrade Your PostgreSQL Version Using Docker” blog post. I was able to accomplish the upgrade with some modifications.

To upgrade PostgreSQL from 11 to 14 (other versions seem to require the same steps), these general steps must be accomplished:

  1. Create a temporary folder for the database backup and share it with your PostgreSQL container instance
  2. Backup the database
  3. Shutdown the database container
  4. Remove the database data
  5. Change the PostgreSQL version
  6. Change password encryption
  7. Start the database container
  8. Restore the database
Continue reading

#docker, #docker-compose, #postgresql, #upgrade

“i” is for Intel on AWS EC2

Last year, AMD-based instances came into existence on Amazon Web Service’s (AWS) Elastic Compute Cloud (EC2). AMD brought a slight performance decrease and a reasonable price discount. CF Webtools is mostly website focused, and almost all of our servers have no problem going with that decreased performance metric.

Since EC2 was created, they’ve used abbreviations, such as “M1” for general purpose (think Main) and T1 for burstable (think Turbo). The characters are the instance class, and the numbers are the generations.

Then in 2020 came the AMD, such as M5a, which appended the “a”. This year came Graviton, powered by Arm-based processors, such as M6g, which appended the “g”.

From day one, any abbreviation that lacked the last character was an Intel processor. The sixth generation has changed that. Now you will start to see “i” appended, if it runs Intel, to flow with the “a” and “g”.

There are other characters in these abbreviations, but that’s for another day.

On August 15th, 2021, AWS introduced M6i instances.