Introduction
In the ever-evolving landscape of iOS app and game development, the approach to testing has undergone a significant transformation. In the past, the "Waterfall" methodology ruled the roost, with testing as a distinct phase. Today, organizations have shifted towards iterative and agile processes that prioritize rapid feedback and flexibility. This shift has brought automated testing to the forefront, revolutionizing the way we ensure product quality. In this article, we'll delve into the key differences between manual and automated testing and explore the benefits of automated testing within an agile development framework.
Waterfall vs. Agile: A Paradigm Shift
Waterfall Methodology:
In the traditional Waterfall methodology, the development process adheres to a linear sequence of phases, including requirements gathering, design, implementation, testing, deployment, and support. Requirements are often rigid and set in stone at the project's outset. Testing, primarily manual, occurs towards the end of the development cycle.
Agile Methodology:
Agile, on the other hand, embraces an iterative and flexible approach. It allows for continuous feedback and adjustments throughout the development process. Unlike Waterfall, testing is no longer a standalone phase but is "Shifted Left" to the beginning and integrated continuously. Automated testing is the norm in Agile, enabling developers to receive timely feedback and adapt to changing requirements.
Benefits of Automated Testing in Agile iOS App and Game Development
Rapid Feedback: Agile's automated tests provide swift feedback to developers, enabling them to detect and rectify issues promptly. This agility is vital in the fast-paced iOS app and game development industry.
Regression Testing: Automated tests ensure that existing features remain functional as new code is added or modified, preventing regression issues.
Enhanced Quality: By consistently running automated tests, developers can identify defects early in the process, resulting in higher product quality. This fosters a culture of continuous improvement.
Flexibility: Agile's adaptability aligns seamlessly with automated testing, allowing for easy updates to tests when requirements change. This flexibility is crucial in dynamic iOS development.
Efficiency: Automated testing significantly reduces the time and effort required compared to manual testing. This frees up developers to focus on coding and creativity.
Test Driven Development (TDD) in iOS
Test Driven Development advocates writing tests before implementing code. It captures the app's behaviour as tests rather than basing tests on code implementation details. This approach ensures that changes and refactoring do not break existing functionality, promoting code robustness.
Red, Green, Refactor
The "Red, Green, Refactor" strategy is a valuable tool in unit testing. It involves:
Writing a test to define behaviour and make it fail (Red).
Implementing code to make the test pass (Green).
Refactoring the code while ensuring the test remains successful (Refactor). This process maintains the required behaviour.
Manual Testing in an Agile World
Manual testing still holds a valid place in the development cycle, particularly for visual and exploratory tests that are challenging to automate. Exploratory testing involves examining the app's behavior outside the scope of specific features exploring and experimenting with available functionalities.
CI/CD: Ensuring Continuous Quality
Continuous Integration and Continuous Deployment (CI/CD) are crucial in an Agile environment. An automated CI/CD pipeline ensures that code changes undergo automated testing before deployment, providing rapid feedback and reducing the risk of regressions.
Our CI/CD Pipeline Example:
Cleaning the workspace.
Checking out the code.
Setting the default scheme.
Building and running automated tests.
Installing certificates and provisioning profiles.
Incrementing version numbers.
Building and archiving the app.
Uploading dSYMs for crash monitoring.
Creating an IPA file.
Deploying to TestFlight.
This automated pipeline guarantees that changes are thoroughly tested and deployed without manual intervention, boosting confidence in the app's functionality.
How to support
This content will be forever free, and if you like it and want to support it, please share it with others. You can also download our games and leave honest reviews. Feel free to get in touch with any questions and feedback, and we'll try our best to respond.
Download Falling Sky from the Apple App Store today: https://apps.apple.com/app/id6446787964
Here's the source code for the GitHub Actions pipeline yaml for Falling Sky:
name: Falling sky CI/CD
on:
push:
branches: [ "main" ]
workflow_dispatch:
env:
ARCHIVE_SCHEME: "Falling Sky"
APP_NAME: "FallingSky"
BUNDLE_ID: com.fallingsky
jobs:
build:
name: Build and Test default scheme using any available iPhone simulator
runs-on: self-hosted
steps:
- name: Clean Workspace
run: |
rm -rf $GITHUB_WORKSPACE/*
rm -rf $RUNNER_TEMP/*
- name: Checkout
uses: actions/checkout@v3
- name: Set Default Scheme
run: |
scheme_list=$(xcodebuild -list -json | tr -d "\n")
default=$(echo $scheme_list | ruby -e "require 'json'; puts JSON.parse(STDIN.gets)['project']['targets'][0]")
echo $default | cat >default
echo Using default scheme: $default
- name: Build for test
env:
scheme: ${{ 'default' }}
platform: ${{ 'iOS Simulator' }}
run: |
# xcrun xctrace returns via stderr, not the expected stdout (see https://developer.apple.com/forums/thread/663959)
device=`xcrun xctrace list devices 2>&1 | grep -oE 'iPhone.*?[^\(]+' | head -1 | awk '{$1=$1;print}' | sed -e "s/ Simulator$//"`
if [ $scheme = default ]; then scheme=$(cat default); fi
if [ "`ls -A | grep -i \\.xcworkspace\$`" ]; then filetype_parameter="workspace" && file_to_build="`ls -A | grep -i \\.xcworkspace\$`"; else filetype_parameter="project" && file_to_build="`ls -A | grep -i \\.xcodeproj\$`"; fi
file_to_build=`echo $file_to_build | awk '{$1=$1;print}'`
xcodebuild clean build-for-testing -scheme "$scheme" -"$filetype_parameter" "$file_to_build" -destination "platform=$platform,name=$device"
- name: Test
timeout-minutes: 30
env:
scheme: ${{ 'default' }}
platform: ${{ 'iOS Simulator' }}
run: |
# xcrun xctrace returns via stderr, not the expected stdout (see https://developer.apple.com/forums/thread/663959)
device=`xcrun xctrace list devices 2>&1 | grep -oE 'iPhone.*?[^\(]+' | head -1 | awk '{$1=$1;print}' | sed -e "s/ Simulator$//"`
if [ $scheme = default ]; then scheme=$(cat default); fi
if [ "`ls -A | grep -i \\.xcworkspace\$`" ]; then filetype_parameter="workspace" && file_to_build="`ls -A | grep -i \\.xcworkspace\$`"; else filetype_parameter="project" && file_to_build="`ls -A | grep -i \\.xcodeproj\$`"; fi
file_to_build=`echo $file_to_build | awk '{$1=$1;print}'`
xcodebuild test-without-building -scheme "$scheme" -"$filetype_parameter" "$file_to_build" -destination "platform=$platform,name=$device" -test-iterations 3 -retry-tests-on-failure
- name: Install the Apple certificate and provisioning profile
env:
BUILD_CERTIFICATE_BASE64: ${{ secrets.DIST_CERT_BASE64 }}
P12_PASSWORD: ${{ secrets.DIST_CERT_P12_PASSWORD }}
BUILD_PROVISION_PROFILE_BASE64: ${{ secrets.PROVISIONING_PROFILE_BASE64 }}
KEYCHAIN_PASSWORD: ${{ secrets.KEYCHAIN_PASSWORD }}
run: |
# create variables
CERTIFICATE_PATH=$RUNNER_TEMP/build_certificate.p12
PP_PATH=$RUNNER_TEMP/build_pp.mobileprovision
KEYCHAIN_PATH=$RUNNER_TEMP/app-signing.keychain-db
# import certificate and provisioning profile from secrets
echo -n "$BUILD_CERTIFICATE_BASE64" | base64 --decode -o $CERTIFICATE_PATH
echo -n "$BUILD_PROVISION_PROFILE_BASE64" | base64 --decode -o $PP_PATH
# create temporary keychain
security create-keychain -p "$KEYCHAIN_PASSWORD" $KEYCHAIN_PATH
security set-keychain-settings -lut 21600 $KEYCHAIN_PATH
security unlock-keychain -p "$KEYCHAIN_PASSWORD" $KEYCHAIN_PATH
# import certificate to keychain
security import $CERTIFICATE_PATH -P "$P12_PASSWORD" -A -t cert -f pkcs12 -k $KEYCHAIN_PATH
security list-keychain -d user -s $KEYCHAIN_PATH
# apply provisioning profile
mkdir -p ~/Library/MobileDevice/Provisioning\ Profiles
cp $PP_PATH ~/Library/MobileDevice/Provisioning\ Profiles
- uses: yanamura/ios-bump-version@v1
with:
version: 1.1.0
build-number: ${{ github.run_number }}
- name: Build and archive app
run: |
if [ "`ls -A | grep -i \\.xcworkspace\$`" ]; then filetype_parameter="workspace" && file_to_build="`ls -A | grep -i \\.xcworkspace\$`"; else filetype_parameter="project" && file_to_build="`ls -A | grep -i \\.xcodeproj\$`"; fi
file_to_build=`echo $file_to_build | awk '{$1=$1;print}'`
xcodebuild archive -"$filetype_parameter" "$file_to_build" -scheme "$ARCHIVE_SCHEME" -archivePath "$APP_NAME" -configuration Release
- name: Install Sentry CLI
run: |
# Check if sentry-cli is already installed
if ! command -v sentry-cli &> /dev/null; then
# Install sentry-cli
curl -sL https://sentry.io/get-cli/ | sh
else
echo "sentry-cli is already installed"
fi
- name: Upload dSYMs files to Sentry
run: |
sentry-cli debug-files upload --auth-token ${{ secrets.SENTRY_AUTH_TOKEN }} \
--include-sources \
$GITHUB_WORKSPACE/$APP_NAME.xcarchive/dSYMs
- name: Create IPA file
env:
EXPORT_OPTIONS_PLIST: ${{ secrets.EXPORT_OPTIONS_PLIST_BASE64 }}
run: |
EXPORT_OPTIONS_PLIST_PATH=$RUNNER_TEMP/ExportOptions.plist
echo -n "$EXPORT_OPTIONS_PLIST" | base64 -d -o $EXPORT_OPTIONS_PLIST_PATH
xcodebuild -exportArchive -archivePath $GITHUB_WORKSPACE/$APP_NAME.xcarchive -exportPath $RUNNER_TEMP/build -exportOptionsPlist $EXPORT_OPTIONS_PLIST_PATH
mv "$RUNNER_TEMP/build/Falling Sky.ipa" $RUNNER_TEMP/build/$APP_NAME.ipa
- name: Upload to TestFlight
env:
API_KEY_BASE64: ${{ secrets.APPSTORE_API_PRIVATE_KEY }}
run: |
mkdir -p ./private_keys
echo -n "$API_KEY_BASE64" | base64 --decode -o "./private_keys/AuthKey_${{ secrets.APPSTORE_API_KEY_ID }}.p8"
xcrun altool --validate-app -f ${{ runner.temp }}/build/${{ env.APP_NAME }}.ipa -t ios --apiKey ${{ secrets.APPSTORE_API_KEY_ID }} --apiIssuer ${{ secrets.APPSTORE_ISSUER_ID }}
xcrun altool --upload-app -f ${{ runner.temp }}/build/${{ env.APP_NAME }}.ipa -t ios --apiKey ${{ secrets.APPSTORE_API_KEY_ID }} --apiIssuer ${{ secrets.APPSTORE_ISSUER_ID }}