ARM Templates vs Bicep vs Terraform vs Pulumi

ARM templates, they are instructions that run against Azure to create resources.  

ARM templates will update settings such as tags when added, deleted or updated using json.

If a dependent resource is called e.g. storage, the ARM template creates the storage and if variables are supplied it shall use them.  Removing or changing the dependency won't update the dependent resource, it will create a new storage resource and point to the new storage. 

Note: Templates don't remove dependent resources if they are pulled from the ARM template. It will not upgrade to the latest instructions. I.e. it does not take the infrastructure to desired state config (DSC).

There is a DSC modules for ARM templates, but i believe this is more for VM's.

Bicep is a domain specific language which is DSC, it's json and way easier to build up your infrastructure.

Bicep will run ARM templates "under the hood" and provide the full desired state.  Bicep is a Microsoft Azure only concept and can be used for private cloud infrastructure.  

Terraform is an open source DSL to build IaC, and it works on AWS, Azure and GCP (and Oracle ...).

Terraform is created by HashiCorp, HashiCorp have an amazing tooling set that uses the Terraform language to build, and changes Infrastructure.  Uses the HCL Language.

AWS CloudFormation is the native way to build IaC on AWS.  Supports json and YAML.

Pulumi is an exciting competitor to Terraform.  Can us multiple languages and easy to write new modules.  Code is compiled and uses appropriate cloud API's to create IaC.   Compare Terraform to Pulumi.  Uses TypeScript/JS, Python or C#.  Pulumi is Open Source, and there is a free tier SaaS offering on Pulumi Cloud.

Common Dev setup could be: TypeScript with Pulumi created using VS code npm for packages.  

Pulumi needs a program yaml file, tells it the language to use and other setup.  2 types of providers: Terraform bridge providers and native provides.  There are over 150 connectors most are Terraform .  Support multiple languages e.g. TypeScript, C#, Python Java, YAML.  Easy to integrate into CI/CD/DevOps.  Pulumi is basically a command line tool that can be called from DevOps including Pulumi Cloud.

Pulumi (Any) vs Terraform (HCL lang) vs AWS: CDK (TS, C#, Py, Go, Java) or CloudFormations (YAML)  vs Azure: ARM/Bicep (DSL).

Nutanix - new to me but looks like VMWare using IaC for multi-cloud basically Terraform but manage and monitor.  Very likely to be wrong here.

Summary:

Use what the organisation uses, Bicep is great if you only are using Azure, and for smaller work/specific DTAP IAC using ARM templates is a well trodden route.  If the organisation wants multi-cloud (do not take multi-cloud lightly) then Terraform and even Hashicorp using Terraform is the way to go.  Terraform and Pulumi are for multi-cloud/run at a higher level than CloudFormation, ARM/Bicep.

Note: Resist multi-cloud at all costs, I've heard ridiculous reasoning like we don't want to be tied to only AWS or Azure.  That's cool but costs go up exponentially with multi-cloud, it's not even the infra that costs, it's the management overhead and testing/verification.  Which parts can be changed.  A business would need a really good reason to go to multiple vendors.

AWS vs Azure vs GCP Comparison

Overview:  I predominately use Azure & Microsoft for all my cloud services.  

Source: https://www.cloudhealthtech.com/blog/aws-vs-azure-vs-google

I have installed multiple SharePoint farms and setups on AWS EC2 instances and I'm currently preparing for the Cloud Practitioner AWS exam.  I have used Google for authentication, SaaS nut not as a IaaS offering.  I'm also a huge fan of Heroku which is great for PaaS and I used this to host my game built for Facebook games.  I've also seen IBM's cloud offering a few years ago.  For me it is too niche and not as feature rich.  So basically I understand Azure's offering well so I found this comparison pretty useful.

My Thoughts:  The contenders:  I really like Heroku for it's simplicity.  I feel for a small Indie developer or company, Heroku has a good free and cheap simple billing options.  GCP, I really can't comment from a good position of knowledge but from what I've used, I like GCP.  GCP is the third biggest Cloud provider.  As a large organisation, I'd only consider the big three: Microsoft Azure, AWS, and GCP to be our cloud partner.  Multi-cloud partner is a demand from some organisations, it's truely extra expensive.  Azure uses ARM templates and has many options for provisioing the IAAS, PaaS offerings.  If you are thinking multi-cloud consider Terraform by Hashicorp for IaC.  There is also the concept of Click-Ops (sic) which allows you to click thru the UI of the management of the Cloud services to get the the desired architecture, this is fine for simple small architecture but you can't do this at any scale or agility and it's super error prone.  Click-ops is more a joke term for the laziest way to build infrastructure and we need to make it sound modern.  IBM's offering, well if you are a partner, you cloud go with this option but it is aimed more a large business partners.  IBM's cloud is IaaS focused, with some PaaS offerings but once again I'm not an expert.

AWS, has always been really easy to use.  It is big and complex like Azure with many offerings.  Basically, I'd choose AWS if the organisation was already using it and the people in the org know have experience with AWS.  AWS originally was aimed at the B2C/startup market but was first to market at scale.

Azure, so in my world Azure and O365 feel like the dominant player but the diagram below provides a great insight into the relative size of the Cloud infrastructure market.  Azure SaaS offering O365/M365 is also huge and hosted on Azure.   Azure security is well thought out and their thinking on BYOK and geo-location appear to be important.  Microsoft offer Arm templates and DSC for configuring environments, they are also adding Bicep which is an abstract layer that will run ARM templates into Azure.

Source: https://www.statista.com/chart/18819/worldwide-market-share-of-leading-cloud-infrastructure-service-providers/

There is good resource CloudWars.co that goes into looking at the various cloud providers.  My current take away is Amazon is the biggest player in the IaaS field.  Azure has IaaS, a large PaaS offering and a massive SaaS (including Dynamics and O365) offering (Amazon has no equivalent).  I am focused on PaaS solutions for my customers so as to remove the infrastructure and process overheads of IaaS.

Off the top of my head reasons for moving and objections I hear for the cloud regardless of platform:

Why Cloud:

1. Save Money
2. More Secure
3. Fast Delivery/More Agile/Easy to scale/Increase business resilience
4. Eco-friendly

Challenges:

1. Lack budget
2. Spiraling costs
3. CAPEX model vs OPEX is business common norm that some business find difficult to switch
4. Resources/Skills
5. Believe security is an issue/Don't trust the Cloud
6. Migrate legacy apps (for me don't move to the cloud unless you get significant advantage)