Learning “AWS Backup” Restorations for On-Prem VMWare VMs

CF Webtools has maintained VMWare ESXi guest OS instances, managed by vCenter, for about 7 years. They are a mix of Linux and Windows Server OSs and are maintained at a secure and redundant co-location data center. While an expensive up-front investment, it has paid for itself over those years, and we have a plan to continue that solution for about another 5 years. A recent upgrade to the next major version proved that virtual machines take a fraction of time for maintenance compared to bare metal instances. Granted, there’s some spin-up time when things work for so long, and you must remember, research, and troubleshoot procedures. Managed cloud takes almost all that time out of the equation, making it my favorite. Though I do miss hands-on hardware here and there.

Some of our on-prem VMs are critical, and some are not. The critical ones have always been backed up with different solutions, depending upon what they are and what the recovery needs look like. However, almost all have come with challenges. So I wanted to look for a VM snapshot-based cloud backup solution that I could trust and would be budget-friendly.

My first direction was to research Veeam. Their solution is very well known. However, it was a struggle to get the attention of Veeam and CDW as a small business without an existing account. I was able to lean on one of our hardware vendors, xByte, who hooked us up with one of their Veeam partners. But it was determined that it was fairly costly with a per-instance license model compared to our existing solutions. So I continued my search.

I then found AWS Backup has an on-prem VMWare solution. AWS Backup is relatively new to the backup game, but its implementations are continually growing. We currently use that service for all our AWS EC2 backups. That service was a “God send” after numerous awful implementations of custom Lambda/CloudWatch scripts and an EBS Automation method. Finally, a solution for what should have been around since the start of EC2.

As of November 2021, AWS Backup offers backup for on-prem VMWare vCenter servers. You must install their Storage Gateway virtual appliance as the “middleman” agent. I was hoping for an “agentless” solution; however, we only pay $0.05/GB-Mo warm storage and $0.01/GB-Mo Cold Storage. That’s a considerable saving, considering we do not have to pay for a license per instance, and there are no incoming bandwidth fees! We will have to pay bandwidth for on-prem restores, but considering that is very rarely done, and bandwidth is relatively cheap, it’s a non-issue. We’d have to pay for storage anyway, so there’s no change.

Another significant advantage is we get a single backup solution for both on-prem and AWS Cloud. It’s one less piece of software we must be familiar with, document, troubleshoot, and keep updated. Outside of an office domain controller, we also anticipate a complete cutover to AWS in 5 years.

Continue reading

#aws-backup, #backup, #vm, #vmware, #vsphere

Wiki.js Active Directory Authentication Configuration

I have recently taken the opportunity to explore Wiki.js as a replacement for a MediaWiki system.

Wiki.js Docs Example

You can add authentication strategies from sources such as Auth0, Azure AD, Facebook, GitHub, Google, LDAP / Active Directory, OAuth2, Slack, and a number more. The current network has a Domain Controller with an Active Directory, so I wanted to integrate that.

The first thing that came to light was that Active Directory (AD) will talk in LDAP on port 389 and LDAPS (secure) on port 636 natively. Sounds great! However, not being an LDAP or Active Directory expert and a lack of documentation from Wiki.js, this became a challenge of “try and sees”.

Continue reading

#active-directory, #ldap, #wiki, #wiki-js

Upgrading PostgreSQL on Docker Compose

I took the dive into Docker Containers on Linux. I’ve been meaning to do this for quite some time, but really my specialty of ColdFusion and Microsoft SQL Server really doesn’t require much in the way of containers.

However, my current project is to convert our internal wiki from WikiPedia to Wiki.js. During this process, I’ve learned about Docker and Docker Compose. I’m currently running Docker 20.10.12 and Docker compose V2. So I use “docker compose” rather than “docker-compose”. This is being run on Amazon Linux 2, on-premise. These steps should work for any RHEL-based Linux distro and beyond.

During this process, I somehow ended up running PostgreSQL 11 when the latest version is 14. I already had data in the database, so I thought it’d just be best to learn how to upgrade PostgreSQL, which is not as simple as changing the version number in the docker-compose.yml config file.

Thank you to José Postiga with Better Programming for getting started with the “How to Upgrade Your PostgreSQL Version Using Docker” blog post. I was able to accomplish the upgrade with some modifications.

To upgrade PostgreSQL from 11 to 14 (other versions seem to require the same steps), these general steps must be accomplished:

  1. Create a temporary folder for the database backup and share it with your PostgreSQL container instance
  2. Backup the database
  3. Shutdown the database container
  4. Remove the database data
  5. Change the PostgreSQL version
  6. Change password encryption
  7. Start the database container
  8. Restore the database
Continue reading

#docker, #docker-compose, #postgresql, #upgrade

“i” is for Intel on AWS EC2

Last year, AMD-based instances came into existence on Amazon Web Service’s (AWS) Elastic Compute Cloud (EC2). AMD brought a slight performance decrease and a reasonable price discount. CF Webtools is mostly website focused, and almost all of our servers have no problem going with that decreased performance metric.

Since EC2 was created, they’ve used abbreviations, such as “M1” for general purpose (think Main) and T1 for burstable (think Turbo). The characters are the instance class, and the numbers are the generations.

Then in 2020 came the AMD, such as M5a, which appended the “a”. This year came Graviton, powered by Arm-based processors, such as M6g, which appended the “g”.

From day one, any abbreviation that lacked the last character was an Intel processor. The sixth generation has changed that. Now you will start to see “i” appended, if it runs Intel, to flow with the “a” and “g”.

There are other characters in these abbreviations, but that’s for another day.

On August 15th, 2021, AWS introduced M6i instances.

Back to Space

There’s one man’s ventures that I like to follow: Elon Musk

A citzen of South Africa, Canada and the US, he leads up SpaceX, Tesla, The Boring Company, Neuralink, and OpenAI. He worked his way from Zip2, X.com which ended up in the hands of PayPal.

A would love to own the Tesla Model S, Model 3, Model X, and Cybertruck. But I’m going to need a raise first.

But what’s most impressive to me is SpaceX. Their quick-turnaround reusable rocket sections have made space travel so much more affordable.

Since the end of the Space Shuttle era back in 2011, we’ve relied upon the Russians to get U.S. astronauts to space at a hefty price tag.

Now, if everything goes to plan, that will all change around May 7th. The date is not official and is just a target at this point. But April, May or June are likely according to Mark Geyer, director of Johnson Space Center. Whatever the date, it will mark the first human space flight by a private corporation, rather than the government. And of course, that leads to significant reduced costs.

But let’s not forget the government is what got them to their starting block. While these are new engines and, well, everything, they didn’t start from scratch.

After numerous tests from empty flights, to automated cargo deliveries to the Space Station, to testing the in-flight abort system, they are ready for the humans.

Boeing almost made the first human flight, but they’ve got some major software issues they’ve got to work out after their last mission just didn’t cut it.

I’m looking forward to this launch date, and I’m sure much of the world will be watching as well.

#elon-musk, #nasa, #space, #spacex

Seeking ColdFusion Systems Administrator

— This position has been filled. Thank you —

Seeking a ColdFusion Systems Administrator for CF Webtools. We are in Omaha, NE and are accepting both local and remote positions.

  • Yes, you may work from home, so pants are optional unless you are video conferencing.
  • Looking for folks legal to work in the US only. (sorry! We still love you world!)
  • Yes, the position is W2 with full benefits. PTO, healthcare, IRA, dental, vision, disability, life, and a positive, encouraging environment.
  • Our operations group consists of 4 team members so far.
  • They spend their days fixing, migrating, managing and upgrading servers.
  • AWS is involved in about 80% of what we do.
  • You will need to be able to find memes that appropriately obscure as inside jokes among your team members.
  • They are on call 24×7. We rotate out weekends and make sure you have enough “you and family time”. But after-hours calls are very minimal. However, you will be responsible for some scheduled “late nighters” for upgrades and migrations. We try and keep your overall hours to about 40 hours a week average still.
  • Plenty to do. Lots of scrambling. Lots of appreciative customers and developers who will see you as a savior if you can fix their problem.

We are looking for someone experienced in ColdFusion. Perhaps you are a CF developer looking to change it up or you are already experienced in JVM tuning, lockdowns and such. This job will involve managing servers (this is not a help desk job) – provisioning, migrating code, upgrading the OS and more. On the ColdFusion side you’ll be handling complex troubleshooting, upgrades, updates, installations and more. But don’t get too hung up on only ColdFusion. We also touch other technologies such as WordPress, Python, PowerShell, MS SQL, My SQL, NoSQL and more. Below are some of the technologies we use and you will need to work with. To qualify, you should know more than a few of these:

  • Linux – For this job you probably need more than just a passing knowledge of Linux. You should be fluent in Linux administration. If you’ve set up some distros, used YUM or other package managers, know how to find stuff on a Linux box etc. you are probably qualified. But the more the better!
  • Windows Server – We have a high percentage of Windows servers. Operations manages backups, patching, migrating, upgrading etc.
  • AWS – about 70% of our managed stack is AWS. If you apply for this job you will be expected to eventually test for an associate certification. Training (online Udemy) and testing are paid for, but you must put in the work to get there. Just like college, except more fun!
  • Java/Tomcat – Our primary stacks invariably include Tomcat/JVM. If words like garbage collection, heap, context, web connectors etc. seem familiar you are on the right track.
  • Networking – you should know your way around a network stack, be familiar with firewall rules, IP addressing, NAT etc.
  • Troubleshooting – you should understand how to troubleshoot issues that arise from CPU, memory or disk constraints and performance.
  • DNS – you should understand DNS zones and record types, how they work, and how to modify them.
  • Web Servers – You should understand how to set up a website in one or both Apache and IIS.
  • Email Servers – We manage several email servers or email relays.
  • Jenkins – More of a “nice to have”. We deploy code through Jenkins from SVN or Git. Ops manages deployments.
  • Nagios (network monitoring) – Also a “nice to have”. We use Nagios to manage an array of uptime alerts from external and internal customers.

About CF Webtools

We are not a staff augmentation company trying to find someone to fling out to a spot in hopes they stick. While you work with customers, we care about developers and work culture. We intend to know you and support you. We strive to create a workplace you enjoy. We are looking for IT specialists that match our culture of Can-do, Caring, Communication and Competency. Here’s some items that you need in order to fit in here.

  • Yes, you will be exposed to ______ (windows/mac) even if you are religiously devoted to ________ (windows/mac). We don’t make the rules.
  • You should be able to work with SVN or GIT and sometimes other source control products.
  • You should maintain positive attitude – We interact with respect and gentle humor. Snark is minimized and encouragement is the order of the day. If you are quirky and self-deprecating that will be a plus and you will love it here.
  • You should maintain and enhance your skills set – you will be given the opportunity to work on lots of code, different versions, platforms, integrations, libraries and SDLC organization and procedure. Every one of these is a growth opportunity. If that has you licking your chops climb aboard.
  • We like balance – Our staff have a full life. They ride horses, snowshoe, skydive, sword fight, play instruments, love dogs, golf, learn languages, rear children, go to plays, like to bake, fish, hunting, equestrian sports, skydiving, guitar playing, dog training, macramé, Golf, racquetball, Mandarin, politics (careful!), family outings, school plays, choirs, baking, snowshoeing, ice fishing, hunting, aquaponics, mudding, and the list goes on. We love it all! We think those things make you a better team member and it makes us want to be around you.

Hopefully this helps explain how we operate enough to pique your interest. If you want to take a shot send your resume to jobs@cfwebtools.com or call (402) 408-3733 ext 109 and ask for Chris. You can try extension 105 and ask for the Muse, but you must get past Rachel so be creative! We look forward to hearing from you!

#career, #cf-webtools, #coldfusion-application-server, #job

Windows 10 RDP Freezing

While connected to remote Windows machines via RDP in Windows 10, the connection freezes after x amount of minutes. Pretty often.

The resolution was to disable UDP.

  1. Run gpedit.msc.
  2. Navigate to Computer Configuration > Administration Templates > Windows Components > Remote Desktop Services > Remote Desktop Connection Client.
  3. Set the “Turn Off UDP On Client” setting to Enabled.

This seems to have appeared when I upgraded to Windows 10 Pro 1903, though my RDP isn’t used very often from this machine. I’ve also seen issues on forums dating back to 1809. You can check your version by going to settings > about.

I noticed that I wasn’t getting kicked off anymore, but the quality of the screen went down, like everything became less crisp or fuzzy.

I have read that using a RD Gateway forces MSTSC to use TCP connections (or TCP/HTTPS) which disables the UDP sessions as well.

Update 1/21/2020:

I read that this was an issue with the graphics card drivers. I updated the drivers on my client, but that didn’t help.

I then moved on to MobaXterm, which I probably should have done long ago. Not only did it solve my issue, but the organization is great and supports far more than RDP. Examples are SSH, Telnet, FTP, SFTP, Serial, AWS S3 and more.

#rdp, #windows-10

How Amazon Uses Explosive-Resistant Devices To Transfer Data To AWS

When CF Webtools needs to migrate a large amount of data from on-premise to the cloud, we order one of these devices to do the job. What I didn’t know is they’re rated to be dropped out of an airplane and take on near-by explosions!

Seeking Sys Ops Wizard!

Photo by Startup Stock Photos on Pexels.com

Want to come work for my team? Seeking a systems administrator, for web-based Linux and Windows infrastructure, to work on my Operations Team at CF Webtools.

  • Yes you work from home so pants are optional unless you are Skyping.
  • Looking for folks legal to work in the US only. (sorry! We still love you world!)
  • Yes the position is W2 with benefits after a short (30 day) trial period.
  • Yes benefits include health care.
  • Health Insurance won’t cover your calls to the Psychic hotline, but you might be able to use FSA for that.
  • Yes there are other benefits – 401k, dental, PTOs, disability, life insurance, and a positive, encouraging environment.
  • Our operations group consists of 3 team members so far.
  • They spend their days fixing, migrating, managing and upgrading servers.
  • AWS is involved in about 80% of what we do.
  • You will need to be able to find memes that appropriately obscure as inside jokes among your team members.
  • Plenty to do. Lots of scrambling. Lots of appreciative customers and developers who will see you as a savior if you can fix their problem.

While a knowledge of ColdFusion is not required it would be a plus. This job will involve managing servers and server instances (this is not a help desk job) – provisioning, migrating code, upgrading OS or Java. Below are some of the technologies we use and you will need to work with. To qualify you’ll need to know at least a handful of these.

  • Linux – For this job you probably need more than just a passing knowledge of Linux. You should be fluent in Linux administration. If you’ve set up some distros, used YUM or other package managers, know how to find stuff on a Linux box etc. you are probably qualified. But the more the better!
  • Windows Server – We have a high percentage of windows servers. Operations manages backups, patching, migrating, upgrading etc.
  • AWS – about 70% of our managed stack is AWS. If you apply for this job you will be expected to eventually test for an associate certification. Training (online Udemy) and testing are paid for, but you have to put in the work to get there.
  • Java/Tomcat – Our primary stacks invariably include Tomcat/JVM. If words like garbage collection, heap, context, web connectors etc. seem familiar you are on the right track.
  • Networking – you should know your way around a network stack, be familiar with firewall rules, IP addressing, NAT etc.
  • Troubleshooting – you should understand how to troubleshoot issues that arise from CPU, memory or disk constraints and performance.
  • DNS – you should understand DNS zones and record types, how they work, and how to modify them.
  • Web Servers – You should understand how to set up a website in one or both Apache and IIS.
  • Email Servers – We manage a number of email servers or email relays.
  • Jenkins – More of a “nice to have”. We deploy code through Jenkins from SVN or Git. Ops manages deployments.
  • Nagios (network monitoring) – Also a “nice to have”. We use Nagios to manage an array of uptime alerts from external and internal customers.

About CF Webtools

We are not a staff augmentation company trying to find someone to fling out to a spot in hopes they stick. While you work with customers, we care about developers and work culture. We intend to know you and support you. We strive to create a workplace you enjoy. We are looking for developers that match our culture of Can-do, Caring, Communication and Competency. Here’s some items that you need in order to fit in here.

  • You should be able to setup multiple local environments on your own dev workstation. You should know words like “Apache” or “IIS”. Yes you will be exposed to ______ (windows/mac) even if you are religiously devoted to ________ (windows/mac). We don’t make the rules.
  • You should be able to work with SVN or GIT and sometimes other source control products.
  • You should Maintain positive attitude – We interact with respect and gentle humor. Snark is minimized and encouragement is the order of the day. If you are quirky and self-deprecating that will be a plus and you will love it here.
  • You should Maintain and enhance your skills set – you will be given the opportunity to work on lots of code, different versions, platforms, integrations, libraries and SDLC organization and procedure. Everyone of these is a growth opportunity. If that has you licking your chops climb aboard.
  • We like Balanced Developers – Our devs have a full life. They ride horses, snowshoe, skydive, sword fight, play instruments, love dogs, golf, learn languages, rear children, go to plays, like to bake, fish, hunting, equestrian sports, skydiving, guitar playing, dog training, macramé, Golf, racquetball, Mandarin, Politics (careful!), family outings, child rearing, school plays, choirs, baking, snowshoeing, ice fishing, hunting, aquaponics, mudding, and the list goes on. We love it all! We think those things make you a better developer and it makes us want to be around you. We aren’t looking for 80 hour a week developers slavishly devoted to coding. We are looking for eclectic, interesting people who enjoy coding and want to do it for a living.

Hopefully this helps explain how we operate enough to pique your interest. If you want to take a shot send your resume to jobs@cfwebtools.com or call (402) 408-3733 ext 126 and ask for the Kurt. You can try extension 105 and ask for the Muse, but you have to get past Rachel so be creative! We look forward to hearing from you!

#job

Getting AWS Java SDK 2.0

In the past I’ve always used REST calls to the AWS API from ColdFusion. There are never any complete CFC libraries that work and they’re almost always dated. The reason being that AWS moves so fast, it’d require a full time person or more to keep it up-to-date and complete.

I am moving towards using the AWS Java SDK to call Java methods from ColdFusion. The SDK is kept up-to-date regularly by AWS and is quite complete and proven. The most common SDK in use today is version 1.x. However, late last year they came out with version 2.0.

According to AWS, “it is a major rewrite of the 1.11.x code base. Built with support for Java 8+, 2.x adds several frequently requested features, like nonblocking I/O, improved start-up performance and automatic iteration over paginated responses. In addition, many aspects of the SDK have been updated with a focus on consistency, immutability, and ease of use.”

But as a non-Java developer that uses Java libraries, this hasn’t come without difficulties. Because of its sheer size, AWS requires you to compile the source into a JAR file. You can compile all of it, which took me 1 hour and 3 minutes at a size of 122MiB. However, they recommend only compiling the (components) service that you plan on using.

I initially installed Maven on Windows 10 to compile it. However, as of version 2.3.6 there is a bug which makes the test fail in Windows, and thus the build. An issue was opened to resolve this and as of 1/22/2019 is pending to be merged into the master branch.

Therefore I compiled in Ubuntu for Windows.

Here’s my commands I used to get the environment ready and build the whole SDK using Maven:

sudo su
apt-get update && apt-get upgrade
# Install Maven
apt install maven
# Install Java SDK 8
apt-get install software-properties-common
add-apt-repository ppa:webupd8team/java
apt-get update
apt-get install oracle-java8-installer
# Verify Maven works and it does not throw a JAVA_HOME notice
mvn-version
# Get the AWS SDK source
git clone https://github.com/aws/aws-sdk-java-v2.git
# Check out a tag containing the release you want to use for the build
cd aws-sdk-java-v2
git fetch && git fetch --tags
git checkout 2.x.x
# Build out the SDK
mvn clean install
# compiles to ./bundle/target/aws-sdk-java-bundle-2.x.x.jar

Now, as I mentioned before, it’s recommended to compile only the components (services) you are going to use to reduce the JAR footprint.

The guide for this can be found here: https://docs.aws.amazon.com/sdk-for-java/v2/developer-guide/setup-project-maven.html

However, I found that guide to be fairly unhelpful. Currently I haven’t been able to get it to build successfully (it creates an empty JAR file).

Basically it’s supposed to use a “Bill of Materials” in the “MVN Repository” as your dependency dictionary. Then I believe it’s supposed to download the source files located in the MVN Repository, based upon your dependency definitions.

Here’s my pom.xml file that is used to define all that:

mvn-no-jar

After hours of frustration, I decided to boot up an AWS Linux 2 instance to see if maybe it was Windows Ubuntu related. Interestingly enough I got a different outcome.

When looking at the contents of the target jar, it looks promising. Not exactly sure what to expect just yet.

#jar, #java, #sdk