adam argyle

Yesterday I presented at the Penn State MacAdmins Campfire Sessions about testing as a Mac Admin. While the video isn’t available yet I’ve written some notes here to accompany my presentation.

Everyone Has a Testing Environment

Some people are lucky enough to have a separate production environment too. There are a lot of things to talk about, but first let me make a few points about testing:

• Don’t test on your daily driver machine. Its tempting, but consider one of the options I’ll detail below to test on a device that doesn’t contain your regular day to day data and configuration.
• Use AppleSeed to test Beta versions of major and minor updates. Testing these things in advance gives you an advantage when looking to deploy at large scale, you will see issues you can resolve earlier
• Create a “Testing Group.” These users can be easy to find, they’re likely emailing you or putting in help desk tickets to update sooner. Utilize those people with an agreement that they test in advance of wide rollout and plan to collect feedback. Real users will often find issues you haven’t and as a result make great testers.
• Plan for testing. If you don’t making time in advance to plan, you’ll be putting out fires when they come up instead of being able to work on them in advance

Testing Management Tools

• New open source tools. Maybe you haven’t tried a tool yet but heard great things. You might want to use Nudge to gently suggest users upgrade to the latest macOS. Maybe you’re just dipping your toes into Mac management and want to test out Munki for software management.
• New features in existing tools. Maybe the latest build of your Mobile Device Management, or MDM, tool includes some new features you want to roll out or test. Apple updates the MDM spec with some frequency, and those might take a bit to be implemented into the MDM you are using.
• New tools from existing vendors. Jamf has a couple features to add on you might not be using if you’re a customer (or even if you’re not) like Jamf Connect or Jamf Protect.

Many paid commercial tools offer some sort of free trial. Many MDMs offer limited support for a small number of devices, feature limited version, or a limited time trial. These are all great options to work on learning new things. Some tools, like Okta for instance, have developer programs so you can learn about the tool even if you’re not yet using it.

Open source products often have a rich community with many blog posts or support from other admins on the macadmins slack. Many are well documented on their wiki pages on GitHub, like the Demonstration Setup for Munki that walks you through setting up a test instance.

A great tool for a beginner with multiple open source tools baked in is Two Canoe’s MDS. Initially a tool to use a USB drive to quickly wipe and reinstall an OS with your configurations and packages, it has become a very powerful tool that now includes MicoMDM, Munki, MunkiReport and tools to download multiple versions of macOS installers. This is a great tool to learn with as it can handle just about anything to manage macOS devices.

Testing macOS

You may want to test macOS for many reasons. New major or minor releases from Apple can change features you were using or add new ones you may need to. There may be software you are deploying and need to check compatibility. You may need to test deploying and installing updates.

You can test devices in many ways:

• Virtual Machines. Virtual machines have some advantages for testing. There are virtualization programs that can be as cheap as free. You can roll back to snapshots to start over again with testing making repeated testing an automated workflow very quick. One drawback is that a virtual machine does not fully replicate the experience on real hardware
  • Local Virtual Machines. Using a virtual machine installed with a program like VMware Fusion, which now has a free version for personal use. There are alternatives such as Parallels and VirtualBox as well. These are considered a Type 2 hypervisor, which means it runs on top of another operating system and a guest shares resources with the host system. There are some downsides as you may need more RAM or storage to run local VMs, and currently macOS guests are not supported on Apple Silicon devices. You may suffer reduced performance on your primary machine while running a VM. Here’s a great guide from HCS on setting up a virtual machine with Fusion Pro 12 and adding a serial number to test automated device enrollment.
  • Baremetal Hypervisor. You can also install a bare metal hypervisor, considered a type 1 hypervisor. This means there is no underlying operating system for the guest OS. They can be very quick and also can be free with VMware ESXi. This requires dedicated Apple hardware but will run on most Mac minis. There are caveats with newer systems so a 2012 or 2014 Mac mini may be the best low cost option (especially if you have an old one lying around).
• Physical Test Devices.
  • startosinstall --eraseinstall. Added in macOS El Capitan this allows automation of macOS installation. in High Sierra we got some cool new features like the erase install option. This allows you to quickly erase and reprovision systems for reuse.
  • APFS Snapshots. You can take a physical device and create an APFS snapshot (on High Sierra or newer as it does require APFS support). Here is the original Mod Titan blog post detailing how to use it. Its a great way to quickly test, but there are some limitations when it comes to SecureToken or Volume Owner on T2 and M1 equipped devices. Nevertheless it is a good tool to use if you’re aware of the limitations.
  • Secondary Install on a Different Volume. Apple provides an article on how to create a new volume on an APFS device to install a test OS. This can be applied to your existing system to install a second version of macOS to use for testing without disrupting your primary machine
  • DFU Restore (Apple Silicon devices only). Device Firmware Update is a tool some may be familiar with from iOS devices. It puts the device into a stat where you can quickly load the firmware on or erase and update quickly. It is a little tricky with laptops but Mr. Macintosh has a great blog post with all different models and how to get into DFU and restore. it is very quick once you’ve downloaded the file, and you can download different versions of macOS Big Sur if you need to test updates. Again Mr. Macintosh has a great resource with the download links for Big Sur restore files.

Testing Packages and Scripts

Packages are a great way to build installers you can use with nearly any Mac management tool to install software. You may need to build a package for software you’re installing because you need to add a serial number, customize the installation, or its provided in a non-standard way and may need to kick off with a script. There are many tools you can use to build packages, both command line and with graphic interface.

• Whitebox Packages. This is my preferred app for a GUI to build custom packages. It has tabs for the different components of an installer package, including payload to drop files, scripts (and resources to use with scripts). It also defaults to correct permissions and script names which is helpful for those of us who are not always as detail oriented as we should be.
• Jamf Composer. This product can be purchased standalone and does not require you to be a Jamf Pro customer. It does the same things as the other tools to build a package but includes an interesting tool to do a snapshot to see what files might be changed during an install. This is really useful with non-standard installs, but you may need to tread lightly. It tracks all changes and some may be unneeded for theinstallation.
• munkipkg. Munkipkg is a free command line tool to build packages. You create a folder structure and a file with build information which turns it into a package.
• pkgbuild. This is the built in command line tool to make packages, and all other tools are abstractions to use this.
• autopkg/Autopkgr. Autopkg is a great tool to automate downloading and building packages. Autopkgr gives you a GUI to start with if you’re not really sure where to start. Recipes are the bases, and you can pick and choose existing ones, build your own, or override with options. Its a powerful tool and great for automation. If you get comfortable with these and want to test your own, you can build your own server to host these tools. They can integrate easily with Munki or with additional plug ins they can be used to automatically upload to Jamf Pro.

While not strictly the same, a very useful related tool is Suspicious Package while allows you to see whatan installer package is doing. Its great to learn by seeing what something else does.

Scripting on Mac comes in many flavors but some of the most common ones you’ll see are the following:

• Python. Apple still ships Big Sur with Python 2.7 even though it was deprecated officially in January 2020 (Though originally it had been scheduled to sunset in 2015). Python3 is not included in macOS by default but you can install it with Xcode tools. If you are writing python3 scripts I suggest you consider deploying it with a package like macadmins python or relocatable python so as to not disrupt any other installs; and you can be certain what version you are working with.
• Bash. The previous default shell in macOS to zsh, and a very commonly used scripting language.
• Zsh. This is default shell on macOS as of 10.15 Catalina. While very similar in many ways to bash there are differences. Consider reading Moving to zsh by Armin Briegel for a great in depth explanation.

Its very important to remember things like file permissions and ownership when using scripts, and again to ensure you are using a dedicated test system (whether real or virtual).

Testing Profiles

Profiles are the current Apple approved way to deliver settings to managed devices. They can be created in a few ways, but with most MDMs a good practice is to sign and upload them for distribution with the MDM. You can optionally install with a click and approval. Prior to macOS Big Sur they could be installed via the profiles command. Just before my session I saw a fantastic post from Bob Gendler detailing how to find the preferences you want to set with a profile. Two tools I find very useful for creating them are:

• Profile Creator. Easily create and sign profiles before uploading them to your MDM and it includes many templates for different applications.
• iMazing Profile Editor. Another free application to create profiles including many templates for different applications.

Thanks for reading, and I’ll leave you with a quote from The Hitchhiker’s Guide to the Galaxy: “Don’t Panic.”