Continuously deploying from GitHub to Azure should be easy. In this video I discuss 2 issues I found when deploying an API using ASP.Net 5
Here is what I was trying to do:
- I had a simple ASP.Net 5 Web API project.
For this demo, I just went file | New and chose the Web API Template for an ASP.Net 5 project
- I checked it into GitHub
- I went into Azure and created an API app (I could do the exact same demo using a Web App)
- Then I went in and configured continuous deployment in the Azure portal
It should have been simple…..
There are two issues:
#1: Even the most basic ASP.Net 5 project requires at least a B1 Azure instance size. For Free and Shared instances, there will be insufficient space for the build, even though they give you 1 GB of storage.
#2: If you want to successfully deploy an ASP.Net 5 project, even if it doesn’t require the wwwroot folder, you need to include it for Azure to recognise the project as an ASP.Net 5 project that needs to be deployed.
i.e. even for a pure web API project that serves no static content. If you don’t have a wwwroot folder the project won’t be published to Azure, the repo will just be pulled… then the app won’t run. (the project.json webroot setting appears to be ignored)
For the Microsoft Team:
- #1: I would like to be able to create a simple ASP.Net 5 API or Web project and continuously deploy it to Azure via GitHub.. and have it work on the Free instance size.
- I think it is important for developers interested in Azure to have the freedom to experiment without worrying about getting charged
- This may involve expanding the size of the free tier on Azure, or maybe it means making some exceptions about what is counted towards the 1 gb of storage…. but we want a frictionless entry to continuous deployment of ASP.Net to Azure
- #2 The second thing I really want to see is a few more smarts around how we identify if an application is a .Net app and if it needs to be pulled straight into Azure from GitHub, or wether it needs to be compiled and published. It appears at the moment it is looking for the existence of a wwwroot folder to make the decision. Suggestion: should we be checking for the existence of a project.json or a hosting.json and then checking for the webroot setting to assist in this determination? similar to the work being done on default logic around the webroot https://github.com/aspnet/Announcements/issues/94
This is what I consider to be my list of ‘essential’ tools / skills for a serious .Net developer.
An Agile or Lean methodology
If your projects are not being delivered successfully Scrum will either enable you to start delivering software, or highlight the reasons why you aren’t.
The Scrum Guide which contains all the rules of Scrum is 13 pages long. There is no excuse for not having read it.
If you aren’t doing Scrum, or you aren’t doing it well… get some help.
Out of the box, it gives you so much. I find it incredible how many people have TFS but are only using it for Source Control.
Learn both Team Foundation Source Control and Git, and the strengths of each.
Work Item Tracking
Being able to successfully deliver value to the customer while triaging bugs requires efficient work item tracking processes and tools.
Why-oh-why isn’t everyone doing CI?
Check in your code, have a magic fairy build it, test it and tell you if you’ve broken anything – sounds too good to be true AND it only takes 60 seconds to setup!
‘Fail Fast’ is probably my favourite development catch-phrase: If you find out that the code you’ve just checked-in conflicts with some-one else’s code 2 minutes after you’ve checked it in, it’s easy to resolve the issue. If you only find out there is an issue a week later, it’s much harder.
Get your CI fairy to take the code she builds and then deploy it to a server for you.
If every time you finish a task you can send the user a link to the test server that you just deployed to and say ‘tell me what you think’. The customer will LOVE you… and you get feedback when it’s most helpful.
If you are using Azure, continuous deployment is just so damn easy.
Also, I am having a little love affair with a tool called Octopus Deploy http://www.octopusdeploy.com. Check it out!
As a .Net guy… you need to be familiar with Azure.
It’s remarkable how easy it has become to build and deploy web applications to the cloud. In 30 minutes I can create a source code repository on http://visualstudio.com, check in an MVC project and have it continuously deploying to an Azure website with both Release and Staging environments.
Check out Andrew Coates from Microsoft doing continuous deployment to Azure at FireBootCamp.
Azure is so much more than just websites… but as this is a ‘need to know’ list, I’ll force myself to not go on about how awesome Azure VMs and Web Roles are. If you take an interest in the cloud, Azure is a platform that will have you boring all your friends with tales of how cool it is.
Communication and interpersonal skills cannot be overemphasised. No matter how smart someone is, you don’t want them on your team if they can’t work as part of the team.
- You need to know how to take instructions
- You need to know how to provide feedback
- You need to know how and when to push back, and when to give in
- Sometimes, you just need to shut-up
Patterns, Practices and Principles.
This is the stuff I’m REALLY passionate about.
Building a complex application, that is still going to be maintainable after you have added features to it for several years requires some skills. My mission is to increase the understanding amongst .Net developers that the following are essential, not hard to learn, will improve your job satisfaction and your employers opinion of you when you continue to add value to his application well after other projects have ground to a halt.
This is not about swapping your SQL Database for an Oracle one!
There are many, many benefits to Dependency Injection: Here is the major one for me – you need dependency injection to ensure that your complex system is built in such a way that changes to one module does not cause flow on effects in other modules.
Typical on many projects: first 10 features get thrown together in a week, the next 10 features take 2 weeks, the next 10 take a month, and then every subsequent feature takes longer and longer because of the changes that need to be made throughout the project to implement the new features.
Dependency Injection is about ensuring you can continue to add features to your complex project.
Essential Reading: Dependency Injection in .Net by Mark Seeman
I’m not a code coverage guy. I don’t believe every single line of code needs a unit tests.
I believe the following
You NEED to be able to write great tests.
Your application should have unit tests over 100% of the code that requires unit tests.
This is a topic that I can (and do) talk about for hours, but in summary:
I write tests
- for fragile code, or code that is likely to break
for important code that you cannot afford to break
for code that is hard to read: Well written unit tests are the best form of documentation after well written code.
when writing the test provides me clarity about the code that I am about to write (slipped my TDD hat on there for a minute)
- When writing a unit test will save my boss money because it’s faster than the (damn) debugger
One of my favourite rants: It is often faster to write a unit test than to test the code you have just written any other way.
I see this a lot and it drives me crazy: I’m testing the credit card validation of my online store. I want to test the submitted card number against a regex before I submit it to the Bank.
Debugger Guy: I hit F5, wait for the whole solution to compile, the website to spin up and the debugger to attach. I then select several items to add to my cart, click next, enter my name, phone number and address, click next, enter the credit card number and click next, realise there is a typo in the Regex, click Stop, change one character and press F5 again ! aaaaaarrrrggghhh !
Unit Test Guy: Writes a 5 line unit test in half the time it takes Debugger guy to even reach the code, gets feedback in milliseconds, provides great documentation about what the regex is validating and ensures that changes to the code in the future do not invalidate the current requirements.
Essential reading: The Art of Unit Testing by Roy Osherove
My session on unit testing: Eat your vegetables! Baking Healthy Projects with Unit Testing and Dependency Injection
Common patterns like Repository and Unit of Work
Patterns are important.
They provide a way for developers to communicate about common problems and suggested solutions.
I hate to re-invent the wheel when smarter people have already come before me and done most of the heavy lifting.
Familiarity with common patterns (the good and the bad) is essential to any senior developer.
Clean Code & SOLID Principles
If software were disposable, I wouldn’t be writing this post.
If I could write software today, and then walk away and never have to look at it again I may not care about the day next year when I have to come back and make sense of it – but I do.
Because I have to maintain my code – I should very much care about making it easy to maintain. This is where the idea of ‘Clean Code’ and the SOLID Principles come in.
Essential reading: Clean Code by Robert Martin
Professional developers should be proud of the code they write.
I struggle to understand why most developers still don’t have a good understanding of the above topics. I strongly believe that these areas should be lived and breathed by every member of a development team.
Once again, you don’t have to always use them… but if you don’t have them in your tool-belt you’re selling yourself as a professional carpenter when you don’t even own a hammer.
I’ve deliberately left this until last.
Although most job descriptions are full of technologies, I believe that knowledge of the topics above are far more important.
Here is the current set of technologies that we use by default to build most of our web applications.
- Frameworks and Toolsets-> we like to keep across what’s out there, and being consultants doing new projects all the time we get to try out different things but our default stack at the moment is
- Twitter Bootstrap
- Telerik KendoUI
I deliver large, high performing web applications for enterprises. The processes, tools and technologies above are what I believe an enterprise web developer needs to know. Obviously other development disciplines require very different skills sets.
In many scenarios other tools, frameworks or methods may be more appropriate, but before you start learning all the alternatives you should know and understand the mainstream choice. Then you are qualified to have an opinion about whether something else is better.
If you are working with Azure, you need to check out the Azure PowerShell Cmdlets.
powerful + simple = AWESOME !
Here are a few very simple samples that I use to manage my Azure VMs. (There are Cmdlets to cover all of the Azure features and I’ll cover managing Cloud Services in a separate post.)
Figure: “Get-AzureVM“. It lets you easily see what VMs you currently have provisioned (i.e. Are being charged for).
Figure: The Stop command let’s you de-provision a VM so you aren’t charged for it while you aren’t using it. I schedule this to run on each of my dev VMs.
No more $200 Azure bills for me because I forgot to shut down my large instance !
Figure: Of course you can start your VMs as well.
In case you weren’t sold… here is a list of a few of my other favourites.
Get-AzureRole (List your roles)
Get-AzureService (List cloud services)
Set-AzureRole (Sets the # of instances)
Get-AzureVM (Get VM info)
For more info:
Check out MSDN http://msdn.microsoft.com/en-us/library/jj156055.aspx
I love continuous deployment to Windows Azure from Team Foundation Service! http://www.windowsazure.com/en-us/develop/net/common-tasks/publishing-with-tfs/
Often though we need the flexibility of building and working directly with the WebDeploy package.
The article discusses setting up Azure VMs to run Active Directory, as an alternative to using ‘Windows Azure Active Directory’
Key points in this article
– “To extend AD services such as directory and authentication to VMs in Azure, an architect can now start to include Domain Controllers (DCs) and Read-only DCs (RODCs) in Azure as part of a design or solution.”
– “Microsoft lets you BYON (bring your own network) into Windows Azure, so it’s technically feasible to securely connect on-premise, WAN, and private cloud networks with Azure virtual networks.”
Reasons to have a DC or RODS in Azure
1. Latency: Latency in the AD authentication between on-premise and cloud networks can cause timeout issues (e.g. authentication timeouts) and issues for demanding applications.
2. Resiliency: Having a DC/RODC in Azure ensures the cloud environment continues to function if the connection to the on-premise DCs fail.
3. Cost: “Azure download bandwidth charges are saved by keeping AD-related network traffic such as DNS and LDAP in the cloud. There is no charge for uploads into Azure, so an RODC in Azure, which has no outbound replication channel, will save money compared to having Azure VMs use the Azure virtual network for all AD traffic.”
4. You need a stand-alone AD: “A self-contained AD that lives only in your Azure cloud might provide directory and authentication services to elastic clusters or farms of computers that have no need for authentication with an on-premise AD.”
Create an AD site in Windows Azure
– Azure is really just a huge network of VMs. Configuring AD in Azure is almost the same as hosting AD on VMs on-premise….
“general precautions about ensuring AD recoverability when AD is deployed on VMs apply to VMs in Azure.”
– Issues discussed.. dynamically-assigned network addresses, defining subnets
Provision a DC with the Azure data disk type
– Important details about provisioning a DC:
o “You must add an additional disk to the Azure VM that will be a DC, before running DCPROMO. This second disk must be of the “data” type, not the “OS” type. The C: drive of every Azure VM is of the “OS disk” type, which has a write cache feature that cannot be disabled. Running a DC with the SYSVOL on an Azure OS disk is not recommended and could cause problems with AD.”
o “This means you must not perform a default installation of DCPROMO on an Azure VM, but rather you attach a data disk, then run DCPROMO and locate AD files such as SYSVOL on the data disk, not the C: drive. This link at Microsoft has checklists to add an Azure VM data disk or attach an empty data disk: http://www.windowsazure.com/en-us/manage/windows/how-to-guides/attach-a-disk/”
Alternative Option: “Windows Azure Active Directory” product
– “Windows Azure Active Directory” – is a separate product
– It is an alternative to setting up Azure VMs to run AD (what we are talking about in this article)
– It is an outsourced AD that lives completely and only in the cloud.
– It appeals to Microsoft Office 365, Dynamics CRM Online, and Windows InTune customers
Follow Up Reading
– Guidelines for Deploying Windows Server Active Directory on Windows Azure Virtual Machines http://msdn.microsoft.com/en-us/library/windowsazure/jj156090.aspx
– BYON into the public cloud with Azure Virtual Networks
Microsoft has released the ability to easily create a Virtual Machine hosted on Azure
Figure: I can’t quite believe how easy it is to ‘Quick Create’ a new Virtual machine. It literally takes a few minutes to have a Windows Server running in the cloud.
Figure: Alternatively you create a Windows or LINUX VM from selection of images in the Gallery. You can also add your own images to the gallery. This process only takes marginally longer than Quick Create, but gives you more options for customization.
Situations where I love Azure VMs
1. Have a Dev box in the cloud
I have an Azure VM with Visual Studio 2012 and SQL Express 2012 on it.
I keep it at an Extra-Small instance for most of the month ($15/month) and ramp it up to a Large/Extra Large instance when I want to work on it.
Figure: Changing the size of the VM is amazingly simple. Log onto the portal, choose an option from the drop down and click Save. Wait a few minutes and log back in.
Figure: I love that I can connect to my Dev VM from anywhere…. Including my iPad
Figure: Running Visual Studio 2012 on Azure from the iPad (the Apple Wireless keyboard nearly makes this workable)
Would I pay $233 per month to leave it as a large instance and make this my full time dev environment – No.
2. Training machines in the cloud.
Similar to the above situation, it was great to be able to spin up 24 Medium instances to have people do labs on at the Enterprise MVC course.
Attendees were given RPD credentials and logged into their own Azure VM.
The alternatives we considered were
– Each attendee downloads and installs Visual Studio 2012 RC and SQL Express 2012 on their laptops
– Each attendee brings a laptop running a 64 bit OS capable of running a VM, and downloads a VM onto their machine.
The issues that we had with using Azure VMs to run training machines were
– Medium VMs were a bit slow – next time we should use Large
– RPD is not heavy, but we did have issues in some offices with the number of people connected to the available wireless routers
– The network connection died at one training venue. Luckily this happened late in the afternoon when most people had completed the lab exercises
– Some VMs would simply fail to provision. The VM feature is not actually released yet and I’m sure this will be addressed
– Provisioning the machines takes some time because they have to be done synchronously. i.e. Create a new VM, wait for it to finish provisioning and then create the next one.
This is something we will look at automating ourselves, but it would be great to have the functionality available on the MS portal.
For a single day of training with 24 Medium VMs this cost about $50. This is incredible value.
Situations where I do not think Azure VMs are the answer
1. Hosted Full Time Server
For a full time machine of reasonable size you are looking at $335 or $671 per month. I doubt this is cost effective.
I’d be interested to hear from some Sys Admin types about what they believe their on-premise servers cost them per month.
Azure VM Pricing Summary
|Preview Prices||cores||memory||per hr||per day||per month|
|Extra Small||Shared||768Mb||$ 0.013||$ 0.31||$ 9.49|
|Small||1||1.75 GB||$ 0.080||$ 1.92||$ 58.40|
|Medium||2||3.5 GB||$ 0.160||$ 3.84||$ 116.80|
|Large||4||7 GB||$ 0.320||$ 7.68||$ 233.60|
|Extra Large||8||14 GB||$ 0.640||$ 15.36||$ 467.20|
|Figure: This is the pricing while the VM feature is in preview|
|Release Prices||cores||memory||per hr||per day||per month|
|Extra Small||Shared||768Mb||$ 0.020||$ 0.48||$ 14.60|
|Small||1||1.75 GB||$ 0.115||$ 2.76||$ 83.95|
|Medium||2||3.5 GB||$ 0.230||$ 5.52||$ 167.90|
|Large||4||7 GB||$ 0.460||$ 11.04||$ 335.80|
|Extra Large||8||14 GB||$ 0.920||$ 22.08||$ 671.60|
|Figure: This is the pricing after the VM Feature is released|
Everyone knows about my deep love for Azure. Most people also know it’s a bit of a love-hate relationship.
It’s so full of promise, but everything turns out to be a bit harder to implement in practice than it should be… until now.
Figure: In the Azure portal, go to your Account settings, then to the Preview Features tab, and select ‘Try it now’ next to Web Sites
Figure: Navigate back to the Portal, click ‘New’, select ‘Web Site’ and then ‘From Gallery‘
Figure: A host of great Apps are available. I’m pushed for time and this is a distraction from writing the course I am delivering tomorrow so I will choose WordPress as I am familiar with it.
Figure: Choose the DNS prefix that you wish to use, select your nearest region and click Next
Figure: Enter a database name, select the region, agree to the conditions and click next
Figure: Complete the WordPress installation form and click ‘Install’
Figure: And you are done. I love this confirmation message.
Figure: And as easy as that I am hosting my own WordPress blog.
Thank you Azure.
And Yes. This blog post took three times longer to create than the new site that I am going to have to move it to.
Azure transactions are CHEAP. You get tens of thousands for just a few cents. What is dangerous though is that it is very easy to have your application generate hundreds of thousands of transactions a day.
Every call to Windows Azure Blobs, Tables and Queues count as 1 transaction. Windows Azure diagnostic logs, performance counters, trace statements and IIS logs are written to Table Storage or Blob Storage.
If you are unaware of this, it can quickly add up and either burn through your free trial account, or even create a large unexpected bill.
Note: Azure Storage Transactions do not count calls to SQL Azure.
Ensure that Diagnostics are Disabled for your web and worker roles
Having Diagnostics enabled can contribute 25 transactions per minute, this is 36,000 transactions per day.
To Do: Confirm that this is per Web/Worker Role
Figure: Check the properties of your web and worker role configuration files
Figure: Disable diagnostics for both roles
Disable IntelliTrace and Profiling
Figure: When publishing, ensure that IntelliTrace and Profiling are both disabled.
Search bots crawling your site to index it will lead to a lot of transactions. Especially for web ‘applications’ that do not need to be searchable, use Robot.txt to save transactions.
Figure: Place robots.txt in the root of your site to control search engine indexing
When deploying to Azure, the deployment package is loaded into the Storage Account. This will also contribute to the transaction count.
If you have enabled continuous deployment to Azure, you will need to monitor your transaction usage carefully.