Saturday, August 23, 2025

Setting up a self hosted Wordpress blog for (almost) free

I recently decided to make a new site to share and showcase my vacations, https://myawesome.vacations! I wanted something a little more modern, and in depth, than these blog posts here on brentsowers.com (which are hosted on blogger.com). After researching options out there, Wordpress is by far the most widely used system to make your own sites - from simple blogs to much more complex stores.

What is Wordpress? 

Wordpress is free and open source content management system (CMS). It's basically a full web site building and management system written in PHP for you to manage the content. It has many plugins that are easily added. Estimates range from 25-40% of all web sites use it! Basically - you can download the source code for the skeleton to start your site with the management console for free. It needs to be hosted somewhere though - it's a full web application that needs a web server and database (which... I'm going to explain how to run a site without that). Because of that, you need it to be hosted somewhere.

Wordpress Hosting Options

There are a LOT of options out there for hosting a Wordpress site. Let me briefly list a few that I looked at:

  • Wordpress.com - essentially the default choice. They have a lot of options here. There is even a free option, although you can't use your own domain name - it's yoursubdomain.wordpress.com. The "personal" option is cheap ($4/month as of July 2025), but does not allow plugin installation. The cheapest option that allows plugins is the business plan at $25/month (as of July 2025). So I discounted it because I want my own domain and want some more control than it gives.
  • EasyWP - This is a hosted Wordpress solution from namecheap.com (who I use for my domain names). So I looked a bit at them. It appears like a good solution - you can easily use your own domain (easier if it's from namecheap). The cheapest option is $6.88/month (as of July 2025), and you get a decent amount of storage. There is a limit on traffic - you only get 50,000 visitors/month, but if you are just launching your site, that's probably not going to be an issue. The reason I didn't use them is, after reading up online about them, the site performance is terrible if you get heavy traffic. It's fine with low traffic, but the site takes seconds to load if you get a lot of traffic. 
  • Amazon Lightsail - Because I already use Amazon web services (AWS) for my other site - coordinatecommons.com, I looked around at what AWS offers. There is a whole service to set up and host a Wordpress site. Full dislclaimer - I didn't actually try setting one of these up, I'm getting my info from what other videos and posts show. Once you have your AWS account set up, just go through the guided process to create a Wordpress instance. But there are a good number of pretty technical things it looks like you need to do - which isn't an issue for me, but may be for others. It's $5/month (as of July 2025), and as far as I can tell, you can install your own plugins - you can do whatever you want with the source code, or just use the management interface in Wordpress. So I was going to use this, but I thought, why am I going to spend $60 every year on a site that will probably get virtually no traffic?
  • Static site hosted in AWS S3 and Cloudfront - this is what I am going to describe in this post. It's almost free, but requires a LOT of technical expertise which is why I'm posting this. This has a lot of disadvantages - it requires a lot more work to set up, you can only do edits on your local computer, and every time you make edits, it takes manual work beyond just editing. You're also limited in what functionality can be on your site. It's not for the faint of heart. But it only costs a base of $0.50/month, with costs going up a little if you have a lot of traffic. If you're interesting, keep reading... 

Static Wordpress Sites with Simply Static

With my coordinatecommons.com site - I put the HTML, CSS, and Javascript in an S3 bucket, and use cloudfront through AWS to host it. That ends up being essentially free because it gets almost no traffic. So I thought, why can't I do this with Wordpress? I don't actually need a live database for my site - it's going to be all static content, no ordering of anything, no forms, etc. 

So I found Simply Static - a plugin for Wordpress that will export a static site to a zip file. You can then upload this zip file contents to an S3 bucket, and voila, you get a static Wordpress site! Once you register on the site, you can download the code for the plugin for free, and just add it to your Wordpress site. They do have pay options to automatically export to AWS, GitHub, Cloudflare, and others, but that starts at $99/year (as of July 2025). Again, because I have no idea how much traffic I'll get, or if I'll even keep adding to the site, I don't really want to spend on the site. So I am going to use the free plugin. 

Step by step guide for setting up a static Wordpress site

Making your own static Wordpress site for almost free does take some work and is definitely not straightforward. Full disclaimer, I am a software engineer with a lot of experience using AWS. If you don't have much AWS experience and don't understand a lot of the terminology I use here, you probably don't want to go this route. For those who are comfortable with AWS, here's exactly how I set it up.

1. Get Wordpress running locally

To build your site to export, you need to run a Wordpress instance on your local computer. I found this site - https://www.hostinger.com/tutorials/install-wordpress-locally - gives a really good overview of how to do it so I won't go in depth. I'm running Windows, and went the XAMPP route.

  1. Download and install XAMPP, and uncheck Perl. 
  2. Then when it starts up, start Apache and MySQL. 
  3. Then follow the steps in that page. That page explains so much I won't attempt to re-explain it. But at this point, you should be able to load up http://localhost/yoursitename/wp-admin to log in, with the username and password you set up during the installation.

2. Build your site

So now, build your site! I don't have much advice here - I didn't spend too much effort in to building a nice site. Here are a few things I did try that you may want to consider for making a nice site:

  • Install basic Wordpress themes - on the Themes tab you can find a lot of Wordpress themes. These met my needs for a basic blog.
  • Elementor - this is a very powerful full site builder plugin, with a free version that does a lot. I was able to install this by downloading the free version, adding it as a plugin from the Plugins tab. But it was a bit overkill for me. Maybe some day I'll come back to this if my blog takes off.
  • Breakdance - another full site builder plugin with a free version, like Elementor. I was also able to install this. Similar to Elementor it was definitely overkill for my needs. 

3. Install Simply Static

  1. Download the free Simply Static as a zip file. 
  2. Go to the Plugins section of Wordpress Admin on your local machine
  3. Upload the Simply Static zip file you just got. Activate it when prompted.
  4. Go to Simply Static on the left menu bar of the Wordpress admin, and click Export. This will take some time.
  5. When it finishes, you can click the Download Now button to get the zip file.

4. Set up S3 bucket

  1. From the AWS console: Go to S3
  2. Click to Create a bucket.
  3. Name it the name of your domain name (this isn't really necessary but makes it easier to distinguish it).
  4. Click the Permissions tab:

5. Click the Edit button by Block public access (bucket settings)

 

6. Then uncheck Block all public access, and click Save.


7. We're not done yet. From the Permissions tab, click Edit by Bucket policy

8. You'll need to paste in this block of JSON in to the Policy area, replacing mydomainname.com with your actual bucket name:

{
    "Version": "2008-10-17",
    "Statement": [
        {
            "Sid": "AddPerm",
            "Effect": "Allow",
            "Principal": {
                "AWS": "*"
            },
            "Action": "s3:GetObject",
            "Resource": "arn:aws:s3:::mydomainname.com/*"
        }
    ]
}

 

9. Then click Save Changes.
10. Now click on the Properties tab. 

11. Scroll all of the way down and there is an option for Static website hosting. Click the Edit button

12. Now click to Enable, and enter index.html as the Index document, and click Save Changes

5. Upload files

Now you can upload everything! You can either manually do this, or do it through the command line. See "Copying files through a script" later on this page for how to do it through the command line. To do it manually:

  1. Unzip the zip file you downloaded from Simply Static to a folder.
  2. Click to Add Files from your bucket in the AWS console, 
  3. Navigate to the directory you unzipped the zip file to
  4. Pick index.html.
  5. For each unzipped folder, click Add Folder, and pick it. Unfortunately I can't figure out a way to upload everything at once.

Once all folders are uploaded, click the Upload button. This will take some time because there will be a lot of files.

6. Test out the site

OK, now you can actually load the site! But... not on your domain name, and not over https. But you should try the site out to make sure it works. Open up the S3 Properties tab for your bucket, and open up the URL that is at the Bucket website endpoint under Static website hosting. Your site should load - click around and make sure everything looks good.

7. Setting up the https certificate

If you've already purchased a domain name - great! You don't need to have purchased it from AWS's Route 53 service. I use Namecheap. AWS does provide free SSL certificates though - you do not need to purchase that from Namecheap. I'll explain how to set up an SSL certificate for your domain name.

  1. Search AWS Console for Certificate Manager and open that up.
  2. Click to Request public certificate
  3. Enter your domain name in, and click Request
  4. When you do this, it will say that the certificate was successfully requested but further action is needed. Go ahead and click the View certificate button. 
  5. Now, you'll need to perform DNS verification by adding CNAME records from your domain name provider. https://docs.aws.amazon.com/acm/latest/userguide/dns-validation.html explains how to do this from the Amazon side - each domain name provider will be a little different for how ti do it on that end. For Namecheap - go to your dashboard, click Manage on your domain, click Advanced DNS, and click Add New Record under Host Records.
  6. From the AWS end - scroll to the right under your domain to see the CNAME name and value. Copy the CNAME name and chop off the part that has your domain name - just paste the first part in to the CNAME host value on your domain name provider. 
  7. Then paste the CNAME value from AWS to the Value in your Domain name provider.
  8. Save the new CNAME on your domain name provider 
  9. After a minute or so, if you reload your certificate details in AWS - the status will show Issued. This means you're good! 

8. Cloudfront to serve content

AWS Cloudfront is used to serve up your S3 bucket through your domain name, through HTTPS. 

1. Find Cloudfront in the AWS Console, and click Create distribution.
2. Enter a name for Distribution name (I don't think this name matters). You can't enter your domain yet if you didn't use Route 53, so skip this for now.
3. Under Specify origin, click Browse S3, and pick your bucket.

4. For Enable Security - only select Enable if you want to pay for it. I think it costs about $8/month for the bare minimum. Since this is simply a static site, I don't think WAF is necessary. Disable it and click Next.
5. Then click Create Distribution.
6. Now you must go back and enter your domain. Under the general tab, click Add domain.
7. Type in your domain name and click Next.
8. Your certificate should be here on the next screen and selected, so ensure that's picked and click Next. 
9. Now click Add Domain.
10. One more setting - form the General tab, click Edit, and type in index.html as the Default root object.
You may also want to pick a lower price class to save a little money.

9. Configure Route 53 to route traffic from the custom domain

We're almost here, this is the last part! Even though you don't need to use AWS Route 53 to purchase the domain name, you need to use it to route traffic to the cloudfront distribution. A way to get around this is to use www as a CNAME entry on your domain name - point this to the cloudfront distribution URL, and then add a URL redirect at your root level to www. But, in my case, I don't want to have to use www, that's an outdated concept. You can instead use Route 53 to manage all routing to allow direct navigation to the root URL (yourdomain.com) instead of having to go to www.yourdomain.com.

NOTE that this is the only part that has a fixed cost - it's $0.50/month. If you really want to get around that, you can use www as the CNAME, but I figure paying 50 cents per month is fine.

I found out how to do this at https://benjamincongdon.me/blog/2017/06/13/How-to-Deploy-a-Secure-Static-Site-to-AWS-with-S3-and-CloudFront/

Here are the steps with updated screenshots:

1.  Find Route 53 in the AWS Console
2.  Click Get Started
3. Select Create hosted zones and click Get Started

4. Enter your domain name and click Create hosted zone

5. Scroll to the right of the Records list and you'll see 4 items listed in the NS record. Copy these (do NOT just use what's in this screenshot - your name server values may be different!


6. Now go to your domain provider and enter these as the name servers for your domain. I'll give instructions for Namecheap:
7. Click Manage on your domain name
8. Pick Custom DNS under Nameservers
9. Enter each of these 4 entries and click the green check

10. Now go back to AWS Route 53
11. Click Create Record

12. Leave the Record name field blank
13. Leave Record type as A
14. Toggle the Alias slider on
15. Click "Alias to CloudFront distribution" under "Route traffic to" 
16. Click "Choose distribution" and pick your distribution
17.  Click Create Records

You're done! You should be able to load your site at https://yourdomainname.com after a bit of time. They say it can take 48 hours for DNS changes to propagate - that's a bit long, but it doesn't take effect right away. Give it a few hours to a day.

(Optional) - if you want to also be able to load https://www.yourdomainname.com:

18. Click Create Record
19. Enter www as the Record name
20. Toggle Alias on
21. Pick "Alias to another record in this hosted zone" under "Route traffic to"
22. Click "Choose record" and pick your domain name
23. Click "Create records"

10. (Optional) Email forwarding

I haven't set this up yet. I've read a few posts how you can set up an SES lambda to SNS topic to lambda which will forward to another address like Gmail, but I haven't set this up.

Backing up database

It's VERY important that you back up your SQL database frequently. I've had a lot of problems with the database randomly crashing (I'll get more in to that in a bit), and since you're running all of this on your local computer, you'll lose everything if your local database gets messed up. 

To back up:

1. Load up phpMyAdmin (click the Admin button from Xampp control panel by Apache, and then click phpMyAdmin at the top)
2. Click on the databae for your site along the left panel
3. Click the Export link along the top menu bar
4. Click the Export button.
5. This will download a .sql file named for your site
6. Copy this file to a cloud backup. I am using my Microsoft OneDrive to do this but any cloud backup will work as long as it's not on your local machine.  I recommend making a new backup file every time - don't keep overwriting the file in the cloud - name the file for the date. This way you can go back to specific dates

Editing Your Site

Once you have your initial content up - you'll certainly want to make changes to it - edits, more posts, etc. To do this:

1. Start up MySQL and Apache from the XAMPP Control Panel application on your local machine
2. Open up WP Admin locally by going to http://localhost/yoursitename/wp-admin/
3. Make all of your edits, and save and Publish.
4. Open up SimplyStatic on the left panel of WP Admin
5. Click Export.
6. Click to download the zip file.
7. Follow the steps in "Upload files" above.
8. Open up AWS Console
9. Go to Cloudfront
10. Click on your distribution
11. Click the Invalidations tab.
12. Click Create Invalidation
13. Enter /* as the path
14. Click Create. Now it will run and invalidate caches - this will clear out all cached copies of your content at all of the distributed versions. So now when you load your site - you should see the new version.

Copying files through a script

Having to manually copy all files through the AWS web site, one folder at a time, is really annoying. It takes a long time and is prone to forgetting something. You can do this through a few command line commands from your computer, but it does take some setup. Let me walk you through this:

Setup 

1. Follow the steps on https://docs.aws.amazon.com/cli/latest/userguide/getting-started-install.html to install the AWS CLI on your computer.
2. Go to the AWS console web site, and search for the IAM service
3. Click to Add User
4. Name it for your site, and click Next
5. Pick Attach policies directly, and pick AmazonS3FullAccess and CloudFrontFullAccess, and click Next
6. Click Create User
7. Now open up the User and click the Security credentials tab
8. Click Create Access Key
9. Pick Command Line Interface (CLI)
10. Leave this tab open!
11. Now run the command prompt from your computer - click the Start button in Windows and type cmd, or on Mac run Terminal
12. Run aws --version to ensure the AWS CLI is installed
13. Run aws configure
14. Copy the Access key from step 10 and paste it here, and press enter
15. Copy the secret key from step 10 and paste it here, and press enter
16. Enter your AWS region - us-east-1 is the default
17. Hit enter at Default output format
18. Now test that it connects by running aws s3 ls yourbucketname. This should show a list of files.

Uploading files

When you want to actually upload new files - whether the first time, or after you've made edits, follow these steps:

1. Make sure you've unzipped the contents of simply static to a folder
2. Open up the command prompt
3. cd C:\path\to\unzipped\simply\static\zip\file
4. Run aws s3 sync . s3://yourbucketname --dryrun. The dryrun option will show you everything that would be changed, but, won't actually upload anything. This is a check to make sure you don't accidentally upload things you don't want. Inspect the output and make sure that this is actually uploading what you want.
5. Assuming that looks correct, run the command again without --dryrun. This will actually upload the files.
6. Now you need to invalidate the Cloudfront cache. Open up Cloudfront in the AWS console UI
7. Copy the ID shown in the list for your site
8. Now run aws cloudfront create-invalidation --distribution-id id-from-step-7 --paths "/*"

That's it! Your new site files should all be accessible.

Troubleshooting

Database Won't Start 

A frequent problem that happens to me that MySQL just won't start. I click to start it in the Xampp control panel, and it will start and then quickly stop with the message "Error: MySQL shutdown unexpectedly. This may be due to a blocked port, missing dependencies, improper privileges, a crash, or a shutdown by another method.". Sometimes, clicking the Logs will show an error, but usually, there is no error in the error log.

Every time, I am able to fix it by following these steps:

1. Go to C:\xampp\mysql in Windows Explorer
2. Rename the data folder to data_old
3. Make a copy of the backup folder, and rename it to data
4. Copy the data_old\yourdatabase folder in to the data folder
5. Copy the file data_old\ibdata1 in to the data folder
6. Start MySQL. It should start now.

This frequent error is why making sure you have a backup in the cloud is so important. One of these times, I'm sure that doing these steps won't fix it. If that happens - I would have to re-install MySQL and import the backup file to get my data back.

MapPress map pin and images doesn't load

I've also encountered an issue with the MapPress plugin and static sites. MapPress is a great plugin to allow you to easily embed maps. Everything works perfectly with it running locally. However, when I load the site at my domain name after uploading - the map pin icon doesn't load. This is a simple fix though:

1.Open up C:\xampp\htdocs\yoursite\wp-content\plugins\mappress-google-maps-for-wordpress\images
2. Load up AWS Console3. Go to S3
3. Navigate to your site/wp-content/plugins/mappress-google-maps-for-wordpress
4. Make a folder named images
5. Go in to images, and upload the red-dot.png, loading.gif, fullscreen.scv, and geolocate.svg files from step 1.

Recommended Additional Plugins

I recommend using these additional plugins with your site:

  • WebP Express - this will convert all of your images to the WebP format, which results in smaller sizes than JPG files that you may upload. This helps the page load faster, and really helps with your search engine rankings. A few tweaks that you should make in the settings to make it work with SimplyStatic, under Settings -> WebP Express:
    • Operation Mode should be CDN Friendly
    • Pick Uploads Only - no reason to mess with your theme's images
    • Pick jpeg only under Image types to convert - no reason to optimize the PNGs
    • Uncheck Create webp files upon request? - this won't work through SimplyStatic
    • Check Convert on upload - this will automatically make WebP's when you upload images
    • Check "Alter HTML", and under that, pick "Replace <img> tags with <picture> tags, adding the webp to srcset" - this will make picture tags in the resulting HTML which works the best through SimplyStatic
    • Click Save Settings
      Now click Bulk Convert and run that for all of your existing images. If you've already uploaded a lot of images before adding this plugin - this will take a long time! As it's running you'll see the percentage smaller that the webp files are than your JPGs.

Google Analytics

You probably want to track visitors to your site. Google Analytics is a great, free way to do this, that has been the industry standard for a long time. I'm not going to write up a full explanation of how to do this, because the site How to Install Google Analytics in WordPress for Beginners does a fantastic job at this. Use option 2 - to embed the code in to your header with WP Code.

Additional Tips/Recommendations

1. Set an Alternative Text for each picture in the Media Library. It won't set one by default, but an alternative text will show up while the picture is loading on slower connections, and really helps with search engine optimization.

 

 

 

 

 

 

 

 

 

Tuesday, January 26, 2021

Best options for sharing state between components with React

I'm building a new web app in React, and so far I've been building everything with functional components and the useState and useEffect hooks. This is so much better than building with class based Components!  This worked fine at first but my application is getting a little complex and I now need to share state between components at different levels.

There are many different ways that you can do this now with React. I evaluated several different solutions and decided to write up explanations of the different approaches, with some code examples, and my rationale for why I chose or did not choose that solution.

useState is so simple and flexible, and easy to understand, so I'd like to find an approach that's as close to useState as possible. Ideally just a single line in a component to get the current value and be able to update the value. Local state is really simple and easy to understand, why shouldn't state shared across a few components be so simple?

After my analysis, I found some libraries that almost did what I want, and showed me the power of custom hooks. So I decided to write my own small library, make-shared-state-hook, to make shared state easy. It generates a custom hook for you that you can put in your components with one line, that returns the current value and a setter function just like useState! But don't take my word for how nice this library is. Please read the different options that I evaluated to understand them better so you can make an informed decision yourself for what approach to use.

Options considered and evaluated 


Option 1 - Prop drilling

Prop drilling is a simple approach where you create local state value at a high level component, and pass it down as a prop to children component that need it. This approach is great for simple applications - it's very explicit and there are no other libraries or magic involved - you see exactly where the data gets created and where it's used. I highly recommend prop drilling if you only need to pass state between 1 or 2 levels of components.

But, like my application that I am building, most applications become more complex and prop drilling becomes unwieldy for some situations. If you rely on prop drilling you'll find yourself passing props through several levels that don't actually need it themselves which is messy. And your list of props can become huge in the higher level components as they need to have everything that all of their children will need as an explicit prop. This will cause different developers working in different areas to trip over each other.  See the following code example:

As you can see here - CounterDisplay and SecondCounterDisplay don't actually need counter - yet they have to have it as a prop and pass it to the next component. And this is for just one piece of data - imagine how much more unwieldy this will become as more data is added.

Verdict - Prop drilling is great for just one level, but a poor choice for anything more than one level of passing. So it's definitely not an option for data that needs to be used in different parts of a complex application.

Caveat - I put this first because everyone should understand this approach and use it where possible. Locally creating state with useState, and passing it as a prop to a single level deep child component is definitely the simplest approach, and simple is good. If your components look fine with this approach, use it!  But... don't design your components specifically so you can pass props around. When you hit a point where you have components passing their own props to children and not actually using those props - that's when you need to move away from this approach.

Option 2 - React's context with providers

Context is typically talked about as the alternative to Redux if you really need to share state. It's designed for application wide settings like locale, logged in user, etc. To use context - you create a context object or value to be used at a high level location, and then wrap the highest level of your application that contains all components that need the context data with a provider component.

This approach is designed to make it easy to read the value many places, but setting the value needs to happen at the highest level component that has the provider wrapper by default. So this only makes sense for a few types of data. However, there is a way that you can get around this by making one of the pieces of data in context a function to set the state value. Within the provider, use useState to store the values. Here's a simple but complete example, partially based off of an approach from the React core team.

There's a lot to like about this approach. It's pretty easy to understand when looking at a component. It keeps data separate and isolated. However, I don't like needing to wrap the highest level containing all of the components needing the context in the provider. This seems like it would get unwieldy as you add more different types of shared state. I also don't like the amount of work required to add a new piece of shared state - you must add the provider, and then traverse up and edit a higher level component to wrap it with the provider.

Verdict - I feel this approach would get cumbersome as you add more than a few different pieces of shared data. And this would lead to you putting different pieces of shared state in to a single object (because that would be easier than making a new provider and wrapping another level in something new), making it unnecessarily complex and inefficient. I'd be much happier with this approach than Redux, but, as you're about to see, I feel there are better ways to share state.

 

Option 3 - use-global-hook

While searching for information, I found a really good blog article, State Management with React Hooks — No Redux or Context API, that explained how you could easily write a custom hook to share state. The blog author wrote a library, use-global-hook, that can easily be put in to your application.You simply call the globalHook function once in a shared state file, passing in an initial value and an object with functions to update the data, and then you can use this in components similar to useState! You can put this in as many components as you want and it will work great - any component can update and all components will get the update. Here's a complete example where I'll show how to use it with two separate pieces of data and multiple components that read and write each piece of data:

Verdict - This approach has almost everything I want! But there are a few things I don't like. Having a separate actions object that you have to pass in seems like an unnecessary abstraction, why not just expose the function to set the data, like the blog post shows an earlier version of this library did? You also cannot use simple types, you have to use objects. This is not like useState - I much prefer the simplicity of useState where you can pass anything to save as state. The way it's written seems to encourage a single global state which I am very much opposed to - why keep all state together when you can keep them separate? You can pretty easily have separate pieces of state by calling useGlobal in separate places, but forcing the data to be an object makes this seem counterintuitive.

Option 4 - make-shared-state-hook

The blog post in option 3 and use-global-hook are really close to what I want, but I really want something that works just like useState. From the blog post on use-global-hook, I saw just how easy it is to use custom hooks to accomplish this.  So I decided to create my own library to do this, make-shared-state-hook. Here is an example:

Create each piece of shared state:

And here's a full example using that:

Now this is what I want - something just like useState that I can use anywhere in the application in any number of components, that all come back to the same data. It meets all of my goals!

There is another library that does almost the exact same thing, react-shared-state-maker. I discovered this as I was publishing make-shared-state-hook.  Great minds must think alike I guess?  I still decided to make make-shared-state-hook, because I wanted to make sure the library has no dependencies. make-shared-state-hook only relies on your application having at least React 16.8 as a peer dependency, no other libraries are needed!

Verdict - This approach has everything I want and meets all of my goals so I am using this library. The simplicity and power of hooks is amazing and has has brought back my love of React after suffering through many years of Redux (see more about why I am not using Redux in a later option)

Caveat - just because this is simple and easy doesn't mean you should use it everywhere. Only make state shared between components when you really need it to be. Keep as much in local state with useState as possible and your application will be simpler, less buggy, and easier to maintain. This approach does make it so that when you need to share state, it's almost as easy as using local state. 

Options not seriously considered

 

Option 5 - MobX

I took a quick look at MobX. I think MobX could have some good applications if you don't have React everywhere, and if you have some extremely complex use cases for sharing your data. But in my opinion, it feels too complex for just about every use case. And I don't like breaking away from React for having shared state. So I didn't do a detailed analysis.

Option 6 - Redux

Redux is always an option. It's been around since 2015 and has been battle tested. Because of my experience with it, I am taking Redux off the table. My prior company starting using Redux back in 2015 and built most UI applications with it for 5+ years, and some of the biggest applications were worked on by dozens of different teams over those years. So I got to see first hand (as both a manager managing teams building applications with it and a coder building simple and complex applications with it) just how bad some of the problems with Redux are.

The biggest issue I have with Redux is that it's so hard to wrap your head around what is happening when you're looking at the component code. And if you use react-redux with connect, that makes it even harder to grasp what's happening. Let's say you're looking at the code for a low level component. You want to know where the data for the props comes from. So you have to look at the higher order component which is usually a separate file. Then look at a mapStateToProps function to see what state values it pulls. Then look at the reducer (typically a different file) to see what the state actually looks like. Then look for dispatches across the application that have an action (yet another thing to look at) that the reducer is looking for to see where data is being set. And all of these are typically in different files. Except it all comes back to a single place where you have to tell redux each reducer, and all the data is stored together in a single object. By this point you've probably forgotten the details of the component because you've had to look at code in at least 6 different files, each file with a different concept to wrap your head around. You can't exactly right click and tell your editor/IDE to find usages of the piece of state.

Redux fails at many other things I'm looking for - it takes a lot of code to make a new piece of shared data and use it (compare that to the 2 lines needed for make-shared-state-hook), your reducer code is burdened with having to account for all of the other state being there, bugs can easily slip in when state data changes and it's hard to find what's using it... I could go on for a long time about why not to use Redux. Now I'm sure there may be a few cases where you need a little more than what make-shared-state-hook offers, but, I'm not sure Redux is the answer.

Option 6a - React's useReducer with context

As much as I love how React added hooks in 16.8, I was really baffled by them adding useReducer and dispatch. To me, this is not really an option - you get most of the problems and complications of Redux, but you'll encounter some more problems that Redux had solved long ago.  

About the Author

I'm Brent Sowers, a principal software engineer at BlackSky. I've been writing code and managing software teams working with many different langues and frameworks for 20 years. Check out the links on the top right to find out more about me. And come work with me at BlackSky!

Thursday, October 10, 2019

Allowing developers to fully own quality

As an engineering manager of two sprint teams that had dedicated quality engineers who do all testing and certification (reporting in to a different quality engineering manager), I recently tried something different - I took the quality engineers out of the two teams and placed quality entirely in the hands of the software engineers. 7 months later, the teams are running great! Levels of quality have been maintained, output of the team has stayed the same, amount of automated test code written has by the team has increased, and motivation and engagement on the team has increased. I'd like to share our story of how and why this worked.

Some of you may read that first paragraph and think "Dedicated quality engineers on a team? What do you mean?" or "Quality wasn't fully in the hands of developers before?".  I would have thought the same thing before I joined a company that ran this way. If this is what you're thinking - this post is not for you. You won't be surprised by anything posted here. This blog post is for people on sprint teams where there are dedicated quality engineers who test and certify all work.
 

Prior process


So let me explain how things had previously been.  We do product development in sprint teams. Within the teams, there are typically 4 to 7 software engineers, and 2 to 3 quality engineers.  Within a sprint (typically two weeks), the software engineers write code for user stories, deploy the code to a testing region, and then hand the user story off to a quality engineer to test and certify. The quality engineer will test it, try to break it and find issues (typically by manual testing), perform regression testing (mostly manual due to lack of thorough automation), file bugs for the developer to fix, and back and forth until they have tested everything and any bugs were fixed. Then they certify the user story.  Automated tests are sometimes written by quality engineers in the sprint, sometimes written in later sprints, and sometimes not written at all. Software engineers will typically write their own suite of API level tests for user stories that have API end points, but not always. The work is mostly done on applications that have lived for years, and do not have very reliable automated tests.

New process


The new process that my two sprint teams are following is similar, but all of the work is done by software engineers. These two teams now just 4 or 5 software engineers, and no quality engineers. The software engineers write code for their user stories including all necessary automated tests - unit, API, UI, whatever makes sense. They will then deploy the code to a testing region, and another software engineer (whoever has the most availability that day) will then do any necessary manual testing to certify the user story, similar to what quality engineers would do before.  So... really not too different than before, just without the quality engineers.  There's a little more to it than this around how we plan out what needs to be manually tested, but, I won't go in to all of those details.

My conclusions

After doing this and seeing the great results, I am really convinced that having quality engineers on sprint teams that automate, test, and certify developers code just does not make sense in the vast majority of cases. This is a relic of waterfall, but, doesn't make sense with scrum. There may be other productive roles for quality engineers, possibly even on a sprint team, but that role should not be to test and certify all of the work coming from developers.

So, why is that my conclusion? And what do I have to share for how to execute a change like this?  You can read the why, or skip straight to my recommendations for how to make this work.

Why does having developers test and certify make sense?


Putting quality entirely in the hands of the software engineers leads to many positive benefits for developers, and brings much better efficiency on to the team. Here are many positive factors why putting quality entirely in the hands of software engineers in sprints makes sense.

Better overall team cohesion

 

When everyone on the team has a similar role of a developer, there is a better sense of cohesion on the team. Here are some factors why the team cohesion is better:

Removing the wall between the coder and tester for work

 

When there are two different groups, one group responsible for developing and another for testing, a wall exists. There is a "toss it over the wall" mentality when tasks move in to test. Developers push code out without much consideration of testers time and availability, and consider their part done when they've tossed it over the wall. Developers know testers are there and will catch their bugs, and don't consider enough that testers are not perfect and don't consider the time and rework to fix bugs. Removing that and having developers test and certify each others work as regular practice removes this wall, because the people they are giving their work to to test are also other developers that have their own development work to do.

Unified direction for all engineers on the team

 

When all engineers on the team report in to the same group, there is a more unified direction for the team. There are no longer two different groups on the team with competing priorities from their management. Tough choices that teams need to make, like sticking to a definition of done and rolling over user stories when automation is not done, are easier for the team to make and the manager to assist with.  Compromises/sacrifices that affect quality and/or automation are easier to make with everyone on the same group.

Better shared understanding of the work being done

 

Developers who write application code all have some base technical skills and understandings that quality engineers will not necessarily have - especially in the active area that the team is working on. This means that discussions among developers over details of user stories tend to go smoother and quicker, and less explanations are needed. This will be discussed more in the section on efficiency gains.

More sharing of technical details of implementation and testing strategies

 

Sharing of technical designs with the whole team, and test plans with the whole team, leads to better designs and better test plans, and enables the developer to understand the testing strategy for their tasks. This can still be done with dedicated quality engineers on the team, but there is much less friction when the sharing is done between software engineers - there is a base technical knowledge that software engineers will all have from building the application code that quality engineers who do not build application code will not have.


More efficiency

 

When developers do testing and certification, there are many efficiency gains.

Reduced time spent manual testing

 

The data was very clear that throughout the 7 months, there was dramatically less time spent performing manual testing on both teams compared to before when there were quality engineers, yet the same level (or better) or quality was maintained. This is a huge efficiency gain. I feel this is because developers have a better base understanding of the technical details of the product and are better able to streamline their manual testing to only what is necessary. A common complaint from the software engineers from when quality engineers were on the team, was that quality engineers were testing areas that were not necessary, and developers had to spend time explaining why it's not necessary, or why bugs filed weren't relevant. This inefficiency was really removed with developers doing the testing/certification.

Less time explaining technical details

 

A common complaint from developers with dedicated quality engineers who do not write application code is having to explain technical details of work to quality engineers. But when the testers are other developers, there is much less time needed to explain technical details of their work, since the other developers are also writing application code and have a base technical understanding of the code and product. Less time explaining = more efficiency. A counter argument to this is that the separation is better to ensure better quality, but we did not see this as an issue.

Better utilization of everyone's time

 

Teams are rarely able to continually deliver functionality to testing throughout the sprint at regular intervals. Work tends to get delivered to test later in the sprint. If there are dedicated testers on the team, this leaves time early in the sprint where these testers are not fully utilized. Sometimes testers will work on automation backlog in this time, but when certifying developers work is their top priority it's hard to focus on and be effective at this automation backlog, since developers work can drop on them at any time. With developers certifying each others work, they are fully occupied with their own application code development work in this early sprint low testing time.

More flexibility in sprints

 

When anyone on the team can test and certify a user story, there is much more flexibility. Several big user stories getting delivered to test late in the sprint is no longer as big of an issue since there are many more people who can test. A single tester taking time off., whether planned or unplanned, no longer causes big disruptions.


More and better automated tests

 

With developers writing automation, more gets written overall, and what is written is more strategic and more efficient to write.

Delivering automation in sprint with application code

 

Having a separate group work on automation after the application coding is complete makes it very difficult to deliver automation for new work in sprint. There is just not enough time at the end of the sprint for this. So, either the automation for work is rolled over as a separate user story, or automation is abandoned for the new work, neither of which are good. With developers writing automation for their own work, it's a lot easier to deliver automation in sprint. Developers are able to write this automation along side the application code, and can modify the application code as needed themselves to support automation instead of having to coordinate with others. This also helps developers write better application code. There is much less overhead and inefficiencies having automation be completed with application functionality in the same sprint.

Better and more strategic automation

 

Developers gain more skills by writing both API and UI automation themselves, and think more about how the features they are developing can be written to allow easier test automation. Better and more strategic automated tests get written that have higher value because of the increased collaboration and discussion among developers of what automation will be written, and developers reviewing each others automation code. Since automated tests are software, and pretty complex and hard software at that, developers think of different and innovative ways to test that may not be thought of if this is purely the responsibility of quality engineers who do not write application code.

Less overlap between different levels of automation

 

With the developer writing all automation for their user story - unit, integration, and end to end/UI, there shouldn't be much overlap in the test code. Integration will test what unit cannot, and UI will test the end to end which can't be tested through integration. The developer will also follow testing best practices and put as much in the unit test level as they can - if integration, or UI, tests aren't needed, it won't get written!  This is more efficient than when there is a separate quality engineer writing this automation - there will inevitably be overlap between this automation with two different people. If you have a really good quality engineer automating a lot - they may be going overboard and overlapping with unit tests, or putting a lot in slower integration or UI tests that should be in unit tests.


Increased ownership and engagement from developers

 

With developers assuming testing and certification responsibilities, they will feel more ownership of what they are building, which leads to higher motivation and engagement.

Developers feeling more ownership of quality

 

When quality engineers own quality, developers naturally do not consider quality as much as they should. Once developers are testing and certifying others functionality, they start feeling more ownership of quality in their own work, because they get to experience testing and certification for others work. So, when developers push things to in test, there are less issues overall that testers find because the developers had been thinking about quality all along - both while coding, and in their local testing. This leads to less overhead of development issues being filed, and higher overall quality.

Developers having more knowledge of what the entire team is working on

 

By testing/certifying others work, developers get directly exposed to, on average, twice as much functionality the team is building than they would if they were "just" developing user stories. Because a developer can test anyone else's code, they're going to want to know and understand everything the rest of the team is doing. There are many side benefits to this - developers feel more engaged on the team, are able to help more with others work, provide more input/suggestions for designs and test plans, and for newer developers, get to learn a lot more about the product. All of these factors lead to a healthier and more motivated team.




Recommendations for best chance of success


I'd like to share some lessons we learned on the teams, and practices which I felt ensured that this was a successful change.

Maintain strict separation of responsibilities between the coder and tester of each piece of work

 

There are endless benefits to having a different person certify a developers work, and endless downsides to not having someone else take a look at a developers work. With developers testing and certifying it could be tempting to have a developer certify their own work. Do not allow this to happen unless the whole team discusses as a group and agrees that no certification beyond automation passing is needed. Quality will definitely suffer and many of the gains from having developers test and certify will not happen if developers are certifying their own work

Review test plans with whole team prior to development starting

 

Ensure that test plans are written prior to development starting on a task, and that the whole team reviews these plans. This ensures that the best and most efficient test plans are made, and allows the whole team to feel ownership. This also helps ensure flexibility for who tests a story - if the whole team participated in the test plan review, the whole team will have some knowledge on the work. And it ensures that the developer knows what the test plan is - this will catch potential bugs and rework before they even start coding!

Anyone on the team should be able to test work (other than the developer for their own work)

 

When anyone on the team can test and certify a user story, work is a lot easier to get to done. In standup - whoever is available can test something rather than waiting for a single person to be available. This flexibility opens up many possibilities that are hard when there are a limited number of testers - several large and small user stories can come in to test at the end of the sprint and still be able to be tested.


All developers should spend a similar amount of time on testing/certification

 

It would be very natural to have the developers that know the product the best do the bulk of the testing and certification. And it would be natural to allow the developers who are more interested in testing, or more willing to do the testing work that no one else wants to do, to take the bulk of the testing. Don't let this happen. Many of the gains will not be realized if only certain people are doing the bulk of the testing. This may require a heavy handed, top down approach from leadership on the team to force the work to be distributed - but this must be done or else you'll lose many of the efficiency gains. Not to mention, you don't want any individual devs spending too much time doing manual testing, or they'll slowly transition to a quality engineer as their primary function. Having everyone do testing, even those who don't know the product too well (provided they pair up with or get guidance from someone who knows the product better), or don't want to, is a great way to spread knowledge among the team!

Developers should discuss and plan what automated tests should get written for each story

 

Developers will have different ideas on what automation should be - what level of depth does each type do, and sometimes even whether automation makes sense.  This should be worked out for each story/task prior to development starting, so the whole team (including manager) can settle on what makes sense. I'd recommend doing this as part of technical design.

Leadership needs to be a strong voice of quality on the team

 

Some developers will adapt very well to these new responsibilities, but some will not and will need help to fully embrace the quality role. The rest of the team needs to be voices of quality here, but in particular, lead developers and the manager need to be strong voices of quality on the team.  Stick to principles - make sure test plans are sufficient in test plan review, make sure valuable automation gets written for all developers work on the team. While the whole team owns quality - leadership needs to ensure that enough quality aspects are considered in test plans and automation plans/execution. Some teams might be able to do this without a top heavy approach, but at the end of the day the leadership needs to ensure quality is adequately being planned for.

Consider edge cases and regression scenarios in test plans

 

Developers most likely will not have trouble adapting to testing the main scenarios for others work - after all developers have always tested this in their own work. However they may not have much experience thinking through possible edge cases or regression scenarios. Ensure these are included in test plans - make sure the whole team is throwing ideas out in test case review for edge cases and regression areas to cover. If the test plan writer, or the entire dev team, doesn't know enough to be able to determine areas to test for regression issues - seek out those who do (devs on other teams, PMs, app support, etc) before assuming regression testing areas are not necessary.

Be smart on the level of testing done

 

You want to ensure that the team maintains high quality - but be smart and efficient about the level of manual testing done. Avoid duplicate testing of areas within the sprint by grouping testing together where it makes sense.  The team should come up with a strategy for repetitive testing (like cross browser testing, and localized content testing) that balances quality with efficiency. This repetitive testing is probably not necessary for every individual user story.

Involve product management to look at user stories

 

It helps to have product management take a look at work during the sprint. They are another set of eyes that can catch overall scenarios, and look and feel issues that may get missed. We didn't have a strict process around this but it might make sense to put some process around this. Regardless of whether devs or quality engineers test I think this is a good idea to ensure that what the team is building is what product management envisions.


 Invest in good automated tests

 

Good automation - that has long term value to ensure quality years later, does not have intermittent failures, and can be a source of learning for future team members, takes time to make. Make this investment. Developers who have not written much API and/or UI automation may not automatically embrace this mindset and may write quick and "just good enough" automation. As a team ensure that the right level of automation is written for all work. This will most likely lead to some tough choices - having to roll over stories because not enough automation was written, adding prerequisite work to get some automation foundations in place, or estimates for some user stories to go up. These tough choices should be made though and not shied away from. The earlier you make the investment and stick to it, the more automatic this will become for all team members.

Learn from quality issues

 

If there are quality issues on the team - look at it as a huge learning opportunity for everyone on the team. Ensure that root causes are analyzed - what could have prevented the escaped issues? Are there safeguards that should be there? Is there enough sharing of test plans? Is the full team engaged in test plan review? Everyone on the team should participate in this, including product management, scrum master, and dev manager. Since developers are doing testing and certification, they will be more likely to participate in the discussion than they would be if there is a separate testing group.

Utilize standup to plan for the day to minimize context switching

 

Developers having to test others work adds more things that developers need to do. To minimize context switching, it's best to plan the day out in standup - have the team talk through who will test what. Unless it's the last day or two of the sprint, or there is an urgent production issue to get tested, don't have someone drop what they're doing the moment that a user story goes in to test.

Allow the team time to adapt to these new responsibilities

 

Don't go in from day 1 of a big change like this assuming that velocity and quality will be the same. It will take some time for the team to adapt - every team will be different for how long they need. So for the first few sprints, be conservative in sprint commitments and pull more work in if the testing goes faster than expected and the team adapts quickly.

Sunday, September 9, 2018

Responsibilities of a lead developer

Over the years I've been a lead developer on different types of teams for different types of companies and environments.  The past few years, I've been lucky enough to be able to manage and mentor some great developers to become leads themselves.  At first, I had assumed that everyone would know what being a lead is about, and what their responsibilities are since they had been on teams with leads their whole career.  However, this was definitely not the case.  Every developer came in to the lead role with different ideas of what their responsibilities should be. Despite me discussing the responsibilities with them, they were really surprised at just how many things a lead is responsible for. In order to help set the expectations with developers over what I think a lead's responsibilities are, I put some effort in to documenting what I view as the responsibilities of a lead developer and I'd like to share that with everyone.

This list is based on my experiences of what I've seen work and not work on teams.  It also reflects the types of companies I've worked for - growing companies with frequent new hires, many less experienced developers (including many right out of school), frequently changing projects and priorities, and in general a decent amount of chaos.  If you're at a stable company on a team of very good and experienced developers working on a long running project, and not a lot of chaos, a lead developer shouldn't have as many requirements as I describe here.  But, I don't think that is the norm in our industry.  In the types of companies I've been at, a strong lead developer that takes responsibility of the output of the team is critical for the success of a team.  I have yet to see a team in this type of environment excel if the team tries to distribute most leadership responsibility to all team members and doesn't have a strong lead.

Also, to clarify, "lead" by my definition is not a manager.  For this post, a lead is not the one that is handling performance reviews, administrative issues, and overall career development for the software developers. Many times it can make sense to have the lead developer also be the manager of the developers on the team, but that just means that the person has some additional responsibilities.  They still have the full lead responsibilities too.  And a lead by my definition does not typically handle overall project planning and coordination between teams on multi team projects. The lead should have input here, but, taking those responsibilities on is a little too much for a lead in my opinion. The lead should be focusing on their individual team.

My list is NOT comprehensive, the responsibilities as a lead developer are going to vary and change all of the time, but all come back to being the leader of the team more so than anyone else on the team.  With being the leader comes accountability for the team.  A good lead developer must be an accountable leader. Read http://www.brandonhall.com/blogs/the-buck-stops-here-a-culture-of-accountability-drives-effective-leadership/ for a really quick summary of what that means. The lead (along with the manager) should be held accountable for the output/results of your team - both the good things coming out of the team and the bad things.  "The team" means the entire team - devs, testers, product managers, scrum masters - not just developers.

So here is my list of lead developer responsibilities. For each responsibility I will give some examples of how you can do this.

Ensure high quality work is coming out of team

High quality means that the work that is being produced follows all best practices, and has minimal to no bugs. There is a lot that you can do to ensure this:
  • Full and good code reviews are being done by you - you are the one who needs to be approving code on the team and enforcing this. If your team does not do code reviews, start!  Others on the team should help in the code reviews but you are the point of responsibility. Many code review comments by you should lead to changes by the developer, but, be sure to allow the developer space to reply if he/she disagrees with your suggestions, and be open yourself to backing down on suggestions for change. Hopefully you will learn some from the developers in code reviews as well.  This should occupy a good portion of your time. Don't just look for styling/syntax.  You should be able to fully explain to someone else what the code is doing, how it's doing it, and why it's doing it that way.
    Ensure proper test code is being added while functionality is being developed, not as follow on work after the functionality is delivered. 
  • Ensure developers are doing the proper amount of local testing - sit with them (meaning actually sit by them and have them go through the local testing with you, or if remote, do a video call with screen sharing) and work with them to see this. When you see bugs introduced by developers, sit with them and discuss it to see how the developer can improve to reduce bugs in the future.
  • If a developer continually produces code that does not follow best practices, sit with them and work with them to identify why this is, and what can be done to help them to produce better code in the future.

Ensure developers are at maximum productivity and that the organization is getting maximum value out of them

Most of us work for businesses, and businesses main objective is to make money.  You as a lead play a key part in this so you must be doing what you can to ensure the developers are realizing their potential and are getting as much done as they can (WITHOUT working long hours). Many leads struggle with this because they do not want to have difficult conversations and do not want to ever say anything negative to people, but this is important as a lead.  Managers should be helping with the difficult conversations, but all responsibility for this cannot be deferred to the manager.
  • Make sure each developer is producing enough - if you feel they are not you need to work with them to ensure they realize the expectations, and see if there are blockers/issues that you can help with. Take their experience/background in to consideration for this.
  • Make sure work is planned out well enough to reduce conflict/overlap with others on the team and outside teams. This can be really hard to do, but will have a huge impact on output of the team.
  • Make sure that there is always work lined up for each developer - my guidance is every developer should have at least a full week's worth of concrete work lined up for them and that they need to know what this work is. This will increase their priority on getting work done - if they do not have anything lined up beyond their current work, they will take longer on their current work (I wish this wasn't true but in my experience it is true for most developers). Be explicit on this and don't assume developers will pick up random unassigned work. Anticipate when they are going to run out of work and get ahead of it.  Depending on the environment they can be the ones to pick the specific work, but you must ensure that what they are working on next is figured out. If you have good developers, this is going to be a real challenge, keeping up with them should be difficult!
  • If there are things the developer continually does that slows them down, sit down with them to understand their process, see if you can suggest a better way of doing it, or plan out tooling/framework changes to help make things more efficient

Mentoring and coaching developers to grow

You work with them more than their manager (unless you are their manager), so you are in more of a position to mentor developers than their manager is.  Any way you have of suggesting different/better ways for them of doing their work, sit down individually with them and discuss it. Here are some ways you can accomplish this:

  • Ensure communication is open and that the developer understands what you mean in your communication.  If you frequently get head nods, and "yeah"'s, this might mean people don't understand what you mean and you need to find another way to phrase what you are trying to say.
  • Ensure that every developer has challenging work, depending on their skill level and experience. Watch over time to ensure that they are not continually given non-challenging work.
  • Ensure that developers don't get pigeon holed in to always being the one to do a specific type of work. If one developer always takes a specific type of work, get this work spread out to others on the team. This might mean you have to jump in to a conversation and say something like "Joe, I appreciate you wanting to take this on, but I'd like to get more knowledge of this spread out among the team. Sarah, can you work on this instead?"
  • Make developers uncomfortable. Get them outside of their comfort zone on occasion.


Help developers - unblock them when they encounter difficult circumstances

Don't wait for a developer to reach out to you - if a developer has not made progress on a task for a few days, it might be best to talk to them individually to see why and what can be done.  In general you cannot rely on developers, or for that matter anyone, to ask for help themselves. Most people don't like to ask for help and like to try to figure out things themselves, but in a team setting, people always days figuring things out on their own is not always best for the team.  Granted, the opposite is not good either - if developers are immediately asking for help for others, you'll need to sit down and talk to them and get them to spend some time trying to figure things out on their own first.

If you don't know how to help a developer that is stuck or in a difficult spot, reach out to others, or get the developer to reach out. First share on the team. Then escalate it up - reach out to other leads, other developers in the company, etc.

Ensure work has had the proper amount of design done, and that the design supports long term usability, scalability, future maintenance

When teams don't have a system to share out to do technical designs on work and share them with each other, I've seen two things can happen.  First, if the developer doing the work is good with designs, they'll do decent designs but deprive the rest of the team an opportunity to suggest better ways, and deprive others on the team a chance to learn.  Or, the design just doesn't happen. This leads to systems and code being produced that are not optimal, and will definitely lead to rework.  And will definitely deprive the team of chances to learn.

As the lead developer, you don't have to be the one to do all of the designs yourself, but, you have to ensure that the designs are being done and shared with the team, and need to ensure that the designs are good. Work being done should be designed to properly meet future needs, be able to scale to meet the expected usage needs (and a little more), and not require high amounts of maintenance in the future.  That can be really hard to do in many organizations, because this will typically mean things will need a little more time to develop right now. But that additional time will pay off very quickly.

I feel that the lead should be doing the most difficult designs on a team. If you don't have the technical skills to do this, and you are not brand new to the team/organization, then, maybe you shouldn't be a lead.  When the design decisions are left to the loudest one on the team, or deferred to architects who aren't part of the day to day of the team, the results will not be good. So you as the lead are responsible for this.

Having the technical skills to do good designs also allows you to help better shape designs that others on the team do.  Much of technical designs are opinions and there is no clear right or wrong answer.  So when disagreements arise of how things should be designed, someone has to make the call over which path to take.  This should be you making this call.  Depending on how your company is structured you may also have architects that will do this, but, I have not worked for organizations that have enough architects to go around to do this for every team.  Now, you don't want to come across as a dictator, so if there are design decisions that others take that you disagree with but you don't feel will lead to poor systems being implemented, you may want to back down and let the designer go with what they feel is the right path.

Some process I recommend to follow to achieve this is to create a standard design template, with questions to be answered for every task/user story done.  Some examples of types of question for the template are any data structure changes (like new database columns, data types, etc), if APIs are used describe any interface additions/changes, and high level description of automated testing that will be written. Having this template is not enough though. You need to ensure that it's followed and this will require persistence, because not every developer wants to be bothered with this type of activity. A key part of this design is not just having developers do it, but share it with everyone on the team.  This will lead to better designs, the more people think about a problem, the more potential solutions will be discussed.  This is also a great way for less experienced developers to learn from the senior developers.

Part of this process can be to require the design be done and reviewed with the team before coding starts on every task.  Many leads do not like issuing mandates like this, and in some environments it's not necessary, but if you feel this will help, by all means institute these processes.

Don't let the design reviews stop at just the team! Reach out to other leads, managers, architects outside the team, etc to get more input for sufficiently complex designs. You as the lead will need to be the one to initiate these conversations.

Ensure future work is properly lined up

This means getting ahead of requirements. If product managers are not ready with upcoming requirements for the team, continually try to work with them to get requirements ready. Offer to help. If they are not willing to let you help, or cannot get things lined up, escalate to your manager and/or their manager.  I don't want to prescribe a specific process, but you as the lead need to help to ensure the work is lined up.  Dropping a ton of requirements on a team doesn't mean the team should start working on these right away - work needs time to be thought through, planned, and designed.

Work with product management to ensure that requirements coming in to the team are structured for maximum efficiency in accomplishing high level project objectives

You should be working with PMs prior to planning meetings on this. Always think of better/more efficient ways to accomplish the end goal and suggest this to PMs (and designers if it's a UI heavy thing). Suggest ways to get to an MVP sooner.  Challenge yourself to always do this even if the idea seems weird/unusual.  Product management working on their own to come up with the path to get to the end goal usually does not lead to the best path.  The more input on this, the better.  Feel free to involve others on the team in the discussions too!

Work with product management to ensure that work coming in to the team is sensible and is a valuable use of the team’s development time

If you don't think the work makes sense for the team to do, that it does not seem like it's worth sinking the time in to, talk to the product manager.  You probably aren't the only one thinking of this.  Push them to come up with the business value and explain it to the team.  Organizational politics can come in to play here, but don't shy away from questions because of this  Getting a project that won't provide much value put on the shelf provides huge value for the organization!

Work with product and project management to ensure project objectives are understood by the team

The more the whole team understands the objectives and feels a sense of ownership, the better ideas will be proposed, and the more motivated the whole team will be (more motivation = happier team members = more output).  This isn't just the responsibility of the project manager or product manager. As lead you need to ensure the whole team understands project objectives.

Ensure builds and releases are stable and go smoothly

This can vary a lot based on how the company is structured with devops, SRE, and development teams.  But you as a lead should ensure there is some sort of  continuous integration/continuous deployment system, and that these are kept stable.  If there is no easy/central build and deployment pipeline and system, you've got some work on your hands!

Keeping this stable is crucial.  When it's time to get a developers code pushed out, whether for internal testing, or to production, it must be easy to do.  I've seen builds fall in to disrepair and unstable automated tests cause people to bypass all automated tests.  If you're lucky you'll have someone on your team that really cares about this and will help others when things start to fail.  But, if you don't, you'll need to take this up yourself.  But I don't want to prescribe any specific solutions, different organizations take very different stances on this.

Ensure proper communication and collaboration is happening between different groups (development, QA, product management, project management, managers) on the team, and that relationships are positive

Some ways that you can ensure this is:
  • Work to get more communication to happen between individuals on the team - you'll probably need to continually push developers to go talk to others on the team and not rely on chat rooms too much.  
  • Make sure that all stakeholders who could have an interest are in relevant discussions/meetings. This is not just the job of the scrum master.
  • Interject yourself in communications that appear to be turning negative to keep them positive and focused

Ownership over services/functionality that your team owns

In most organizations, teams "own" different areas of functionality of the applications that are being used in production.  The team isn't necessarily actively working on all of these areas but should be considered the owner.  What ownership means can vary wildly but typically involves investigating and fixing bugs found, monitoring for functionality degrading/breaking, addressing performance/scalability, and planning future major enhancements.

This can be difficult when code is long lived and teams frequently change.  As the lead, you should be the primary point of contact for these areas. So when inquires come in, you need to be able to answer them yourself or point to someone who would know.  If you don't know where to start, that's a good sign that there needs to be knowledge sharing and more documented.  As the lead, it's your responsibility to ensure that there is knowledge spread so there isn't one single person with all of the knowledge of something. For anything that your team owns, if a high priority bug comes in, your team should be able to quickly diagnose and fix, regardless of who is on vacation. If this is not the case, work to make it true.

Taking care of cross team requests

Lots of random things come up from people outside of the team. Ensure that these are given the appropriate priority and are not ignored. Many times, teams view these as distractions, and scrum masters sometimes view it as their job to shield teams from anyone outside of the team. While this may be best for short term project gains, it's not the best for the overall organization. As the lead you can help here by being a primary point of contact to ensure that others outside the team are getting the necessary support from your team, without distracting the individual contributors from their current priorities any time there is any external request.

Ensure team, and individual team members, are continually striving to improve productivity and deliver more

Never settle on a "good" velocity. Always try to push the team to deliver more, and be smarter about how they do their work.

If you feel individuals on the team can do more, push them to do more. Give them more assignments.

Manage upwards to ensure that management above the team is in the loop and able to help

Some ways you can do this are:
  • Escalate team issues to your manager and seek out their guidance/advice. You should not try to always paint a positive picture of everything to management, this will shield issues that they can help with and make it look like you are not doing enough to get the team to improve.
  • Ensure your manager, and any other relevant stakeholders, knows as early as possible when there are risks to missing commitments that the team has made.  The earlier risk is identified, the better action can be taken (maybe no action is taken but people won't be surprised/taken off guard when commitments are missed)
  • Ensure that the manager of every developer on the team knows of that developers progress.  You should be able to give this manager a frank assessment of the developer. Don't try to shield them from their manager, if they are struggling their manager should be involved to try to help. 

Coding

You're probably not going to get a ton of dedicated time to write code. But still try to set aside time to write code and deliver functionality that the team is working on. If you go too long without writing much code you'll lose touch with what the developers on the team are going through. I recommend  that every lead take on at least one development task per work increment (like sprints).

However, it's probably best to not commit to delivering things that are extremely time sensitive, or could put things at risk if it slips a few days. Try to work on relatively independent pieces so if you are not able to get much time for several days it will not impact the team much.

Conclusion

Well, that's a lot to discuss. I hope you found this valuable, whether you are considering being a lead, you are a lead, or you are managing/leading leads yourself like me.  Writing this up and sharing it with my leads has really helped them to grow in to the role and become great leaders of their teams.  If you have any different ideas, or other types of lead responsibilities that you can think of, I'd love to hear it in comments!