How Amazon Uses Explosive-Resistant Devices To Transfer Data To AWS

When CF Webtools needs to migrate a large amount of data from on-premise to the cloud, we order one of these devices to do the job. What I didn’t know is they’re rated to be dropped out of an airplane and take on near-by explosions!

Seeking Sys Ops Wizard!

Photo by Startup Stock Photos on Pexels.com

Want to come work for my team? Seeking a systems administrator, for web-based Linux and Windows infrastructure, to work on my Operations Team at CF Webtools.

  • Yes you work from home so pants are optional unless you are Skyping.
  • Looking for folks legal to work in the US only. (sorry! We still love you world!)
  • Yes the position is W2 with benefits after a short (30 day) trial period.
  • Yes benefits include health care.
  • Health Insurance won’t cover your calls to the Psychic hotline, but you might be able to use FSA for that.
  • Yes there are other benefits – 401k, dental, PTOs, disability, life insurance, and a positive, encouraging environment.
  • Our operations group consists of 3 team members so far.
  • They spend their days fixing, migrating, managing and upgrading servers.
  • AWS is involved in about 80% of what we do.
  • You will need to be able to find memes that appropriately obscure as inside jokes among your team members.
  • Plenty to do. Lots of scrambling. Lots of appreciative customers and developers who will see you as a savior if you can fix their problem.

While a knowledge of ColdFusion is not required it would be a plus. This job will involve managing servers and server instances (this is not a help desk job) – provisioning, migrating code, upgrading OS or Java. Below are some of the technologies we use and you will need to work with. To qualify you’ll need to know at least a handful of these.

  • Linux – For this job you probably need more than just a passing knowledge of Linux. You should be fluent in Linux administration. If you’ve set up some distros, used YUM or other package managers, know how to find stuff on a Linux box etc. you are probably qualified. But the more the better!
  • Windows Server – We have a high percentage of windows servers. Operations manages backups, patching, migrating, upgrading etc.
  • AWS – about 70% of our managed stack is AWS. If you apply for this job you will be expected to eventually test for an associate certification. Training (online Udemy) and testing are paid for, but you have to put in the work to get there.
  • Java/Tomcat – Our primary stacks invariably include Tomcat/JVM. If words like garbage collection, heap, context, web connectors etc. seem familiar you are on the right track.
  • Networking – you should know your way around a network stack, be familiar with firewall rules, IP addressing, NAT etc.
  • Troubleshooting – you should understand how to troubleshoot issues that arise from CPU, memory or disk constraints and performance.
  • DNS – you should understand DNS zones and record types, how they work, and how to modify them.
  • Web Servers – You should understand how to set up a website in one or both Apache and IIS.
  • Email Servers – We manage a number of email servers or email relays.
  • Jenkins – More of a “nice to have”. We deploy code through Jenkins from SVN or Git. Ops manages deployments.
  • Nagios (network monitoring) – Also a “nice to have”. We use Nagios to manage an array of uptime alerts from external and internal customers.

About CF Webtools

We are not a staff augmentation company trying to find someone to fling out to a spot in hopes they stick. While you work with customers, we care about developers and work culture. We intend to know you and support you. We strive to create a workplace you enjoy. We are looking for developers that match our culture of Can-do, Caring, Communication and Competency. Here’s some items that you need in order to fit in here.

  • You should be able to setup multiple local environments on your own dev workstation. You should know words like “Apache” or “IIS”. Yes you will be exposed to ______ (windows/mac) even if you are religiously devoted to ________ (windows/mac). We don’t make the rules.
  • You should be able to work with SVN or GIT and sometimes other source control products.
  • You should Maintain positive attitude – We interact with respect and gentle humor. Snark is minimized and encouragement is the order of the day. If you are quirky and self-deprecating that will be a plus and you will love it here.
  • You should Maintain and enhance your skills set – you will be given the opportunity to work on lots of code, different versions, platforms, integrations, libraries and SDLC organization and procedure. Everyone of these is a growth opportunity. If that has you licking your chops climb aboard.
  • We like Balanced Developers – Our devs have a full life. They ride horses, snowshoe, skydive, sword fight, play instruments, love dogs, golf, learn languages, rear children, go to plays, like to bake, fish, hunting, equestrian sports, skydiving, guitar playing, dog training, macramé, Golf, racquetball, Mandarin, Politics (careful!), family outings, child rearing, school plays, choirs, baking, snowshoeing, ice fishing, hunting, aquaponics, mudding, and the list goes on. We love it all! We think those things make you a better developer and it makes us want to be around you. We aren’t looking for 80 hour a week developers slavishly devoted to coding. We are looking for eclectic, interesting people who enjoy coding and want to do it for a living.

Hopefully this helps explain how we operate enough to pique your interest. If you want to take a shot send your resume to jobs@cfwebtools.com or call (402) 408-3733 ext 126 and ask for the Kurt. You can try extension 105 and ask for the Muse, but you have to get past Rachel so be creative! We look forward to hearing from you!

#job

Getting AWS Java SDK 2.0

In the past I’ve always used REST calls to the AWS API from ColdFusion. There are never any complete CFC libraries that work and they’re almost always dated. The reason being that AWS moves so fast, it’d require a full time person or more to keep it up-to-date and complete.

I am moving towards using the AWS Java SDK to call Java methods from ColdFusion. The SDK is kept up-to-date regularly by AWS and is quite complete and proven. The most common SDK in use today is version 1.x. However, late last year they came out with version 2.0.

According to AWS, “it is a major rewrite of the 1.11.x code base. Built with support for Java 8+, 2.x adds several frequently requested features, like nonblocking I/O, improved start-up performance and automatic iteration over paginated responses. In addition, many aspects of the SDK have been updated with a focus on consistency, immutability, and ease of use.”

But as a non-Java developer that uses Java libraries, this hasn’t come without difficulties. Because of its sheer size, AWS requires you to compile the source into a JAR file. You can compile all of it, which took me 1 hour and 3 minutes at a size of 122MiB. However, they recommend only compiling the (components) service that you plan on using.

I initially installed Maven on Windows 10 to compile it. However, as of version 2.3.6 there is a bug which makes the test fail in Windows, and thus the build. An issue was opened to resolve this and as of 1/22/2019 is pending to be merged into the master branch.

Therefore I compiled in Ubuntu for Windows.

Here’s my commands I used to get the environment ready and build the whole SDK using Maven:

sudo su
apt-get update && apt-get upgrade
# Install Maven
apt install maven
# Install Java SDK 8
apt-get install software-properties-common
add-apt-repository ppa:webupd8team/java
apt-get update
apt-get install oracle-java8-installer
# Verify Maven works and it does not throw a JAVA_HOME notice
mvn-version
# Get the AWS SDK source
git clone https://github.com/aws/aws-sdk-java-v2.git
# Check out a tag containing the release you want to use for the build
cd aws-sdk-java-v2
git fetch && git fetch --tags
git checkout 2.x.x
# Build out the SDK
mvn clean install
# compiles to ./bundle/target/aws-sdk-java-bundle-2.x.x.jar

Now, as I mentioned before, it’s recommended to compile only the components (services) you are going to use to reduce the JAR footprint.

The guide for this can be found here: https://docs.aws.amazon.com/sdk-for-java/v2/developer-guide/setup-project-maven.html

However, I found that guide to be fairly unhelpful. Currently I haven’t been able to get it to build successfully (it creates an empty JAR file).

Basically it’s supposed to use a “Bill of Materials” in the “MVN Repository” as your dependency dictionary. Then I believe it’s supposed to download the source files located in the MVN Repository, based upon your dependency definitions.

Here’s my pom.xml file that is used to define all that:

mvn-no-jar

After hours of frustration, I decided to boot up an AWS Linux 2 instance to see if maybe it was Windows Ubuntu related. Interestingly enough I got a different outcome.

When looking at the contents of the target jar, it looks promising. Not exactly sure what to expect just yet.

#jar, #java, #sdk

AWS Introduces Service Backups

Currently to backup EBS volumes on AWS, CF Webtools, and many other organizations, rely upon either custom Lambda script that can be buggy or a hard to figure out solution that’s deployed from a package AWS supplies.

Today AWS announced a centrally managed and automated solution for backing up multiple AWS services, including AWS: “AWS Backup

product-page-diagram_aws_backup_how-it-works.aafc7b1324fd4d8b52e9fbcd5c95e14529de27c6

This appears to be a backup solution that seeks out tags for different services. But the key here is this appears to be a well-thought-out solution that we don’t have to dick around with to make it work.

Service levels backups are provided for Amazon EFS and Amazon DynamoDB. Service-level snapshots are provided for Amazon RDS Amazon EBS and AWS Storage Gateway.

You can also backup your on-premise data via the AWS Storage Gateway.

I look forward to implementing this service to provide our clients with a much more reliable backup solution.

New AWS Windows and DR Services

49137469 - a word cloud of disaster recovery related items

On January 7th, 2019, AWS released the AMI for Windows Server 2019. This comes a few months after the October 2nd, 2018 release of the new Microsoft OS.

Some highlights include smaller and more efficient Windows containers, support for Linux containers for application modernization and App Compatibility Feature on Demand. It comes under the standard AWS Windows pricing model.

On that note, Windows Server 2008 R2 (SP1) is at end-of-life in about one year, while Windows Server 2012 R2 End of Mainstream support was back in October of 2018. Don’t let technical debt put you into a predicament. CF Webtools operations team can assist you with upgrading your operating system to a more recent version, whether it be the weather tested Windows Server 2016 or the most modern version of Windows Server 2019.

CF Webtools uses CloudEndure to provide disaster recovery (DR) services to our on-premise and data center clients. The service is low impact and gives you the security your organization demands. We investigated multiple DR services and chose CloudEndure to be our primary solution.

Amazon has recently acquired CloudEndure. The company was backed by Dell, Infosys and others. They have been an AWS Advanced Technology Partner since 2016. More details have yet to surface.

If you are interested in Disaster Recovery Services for your organization, please contact us and we’d love to help you.

Estimating AWS EC2 EBS Snapshots

Estimating and understanding what AWS EC2 EBS Snapshots will cost you can be more difficult than you may think.

Here are some key points to keep in mind:

  • Snapshots are not compressed. Therefore your first snapshot will be equal to the GiB used in the source EBS volume.
  • Additional snapshots are incremental. Each incremental snapshot uses pointers, pointing to the prior snapshot’s blocks that have not changed. New blocks are recorded.
  • You can use the AWS Cost Explorer to view past usage. Today is not available. Filter down by “Usage Type Group” and set the value to “EC2: EBS – Snapshots”. Narrow down further by region and/or tag.
    • Usage (GB) are measured by “GB-Month”. So if there are 30 days in that month, multiple the metric by 30 to get that day’s actual usage.
  • As of 12/10/2018, the cost of a snapshot is $0.05/GB/mo

The hard part is estimating the amount of change per snapshot. The most lenient method would be to use a 100% change value. But that’s not practical.

Let’s say you estimate that 3% of your total volume size will be modified per snapshot. Therefore plan on an additional cost of $.15/mo for every 100 GiB of used volume space on every snapshot produced..

How Many Hours In A Month?

When estimating the monthly cost of a per hour service, over the course of a year, you need to know how many hours in a month. For some reason a simple “average hours in a month” Google Search yields unhelpful results.

730.5 AVERAGE HOURS IN A MONTH

There are 365.25 Julian calendar days per year (366 days in a leap year), 24 hours in a day and 12 months.

365.25 days X 24 hours / 12 months = 730.5 hours

So now you can estimate monthly cost for hourly services, such as Cloud Services.

example: $.10/hr * 730.5 = $73.05 average cost per month over a year