Bootstrapping the Craft Application

This video is part of the Reading Craft's Source Code video series and was made possible with the support of ArcusTech, please check them out!

In this video, we’re going to look at the Craft application bootstrapping process, in other words how a web request is handled by Craft, and how it ultimately determines which action to call, or which template to render. In this case, we’re visiting the homepage of the site, so our URI is a blank string, and we expect the request to be handled by rendering the index.twig template.

Every web request that the server receives is going to end up at the index.php file, which exists in the public web folder. And we can see that this is the case by opening up the .htaccess file that Craft ships with. Here we can see that all requests that are not files or directories, or some sort of favicon, end up at this rewrite rule which redirects the request to the index.php file, appending whatever comes after the base URL to the path parameter, p.

So let’s open up the main index.php file and take a look. Now, depending on when you originally installed Craft, the index.php file in your project might look like this, or it might not. This is a fresh install of Craft, but earlier Craft versions shipped with a slightly different index.php file. When you first set up your Craft project, you very likely did so using composer create-project followed by the craftcms/craft repository, or perhaps a starter Craft project, but either way, running that command pulls down all of the files and folders that Craft requires, into your project’s root directory. Now, whenever you update Craft, you’re updating the craftcms/cms repository, so only files in the vendor/craftcms/cms directory actually get updated. Updating your version of Craft has no effect on the files in your root project, which makes perfect sense since you don’t want any of your project-specific files being overwritten. Your composer.json file, for example, lists all of the plugins and packages that your project requires. Similarly, your .env file contains all of your project’s environment variables, the config folder contains your configuration and the templates folder, your hand-crafted Twig templates. So the version of the index.php file that you’ll have, depends on what that file looked like at the time that you set up your Craft project.

As we previously saw, the index.php file is the main entry point for all web requests, and as we can see here, on the surface at least, there’s not much to it. The first thing that happens is that a shared bootstrap file is loaded, which also exists in the root of the project. Bootstrapping generally consists of setup tasks such as performing checks, defining constants, aliases and paths, autoloading required packages and loading configuration files.

Here, for example, is where Composer’s autoloading process kicks off. This is somewhat off-topic, but since we’re so heavily dependent on Composer for managing packages, I find it really interesting to get an idea of how Composer works under the hood. If we open up this autoload.php file, which is generated when you first run Composer, and if we jump into the getLoader method, we’ll see a few interesting things. Firstly, Composer keeps track of every single PHP file that your project depends on, literally thousands of files. And secondly, Composer registers itself as an autoloader, meaning that it becomes responsible for autoloading PHP files on demand during runtime. Fortunately, PHP has something called OPcache, which stores compiled bytecode, known as Opcodes, in memory, just like a caching layer, significantly improving performance when executing PHP requests.

The last thing that happens in this bootstrap file is that the environment variables are loaded in from the .env file using Dotenv. The version of the Dotenv package that Craft uses is also something that has changed over time. Craft currently requires Dotevn version 5, however previous versions of Craft required version 3. I always recommend keeping software up-to-date, so I’d advise you to go check what version of Dotenv your project is using. If you choose to update from version 3 to 5, be aware that how Dotenv is initialised has also changed, so at the very least you’ll need to update your index.php file and the bootstrap.php file, which you’ll find in the craftcms/craft repository.

So we’ve come to the end of this file, which means that we can close it and come back to the index.php file. Now that we’ve required and executed the bootstrap.php file, we can proceed to load and run Craft. The file that we’re requiring here, which returns an instance of the Craft web application class loaded into the variable $app, exists inside the craftcms/cms repo. And you might also notice that this file exists inside the bootstrap directory, meaning that this is a web request specific bootstrap file. Let’s take a quick look at this file, which remember, is part of the CMS repo, meaning that you will automatically receive updates to it. Very briefly, we can see a check for the minimum required PHP version, a check for a required PHP extension and environment normalisation, your average bootstrap stuff, that in this case is specific to web requests. Once this is complete we send back the result of another bootstrap file, which exists alongside the web bootstrap file, but which is shared between both web and console requests. So looking in the console bootstrap file, we can see that it also requires the shared bootstrap.php file. 

Skimming over the comment headers in the shared bootstrap file, we can identify things such as general setup, determining paths, validating paths, loading general config settings, determining if Craft is running in Dev Mode, load Composer dependencies, and finally initialising the application and returning it. The app is going to be either an instance of craft/web/application or craft/console/application, depending on where the request originated – in our case, it originated from a web request. So that rather long-winded process is what’s involved in returning to us an instance of a craft/web/application class, $app, that we can finally run.

The lifecycle process of a web request is rather involved. Fortunately, both Craft and Yii, the underlying PHP framework, take care of a lot of low-level detail for us. So we’re going to look at just a few aspects of the request handling process, and don’t worry if you feel a bit lost along the way, many plugin developers won’t even be familiar with this. But I find it very useful, even important, to have a basic understanding of how requests are handled in Craft.

You’ve probably noticed how seamless I’m navigating through the source code with my IDE, and I’m using PhpStorm which is a phenomenal tool for doing this. Just by holding the Command key and clicking on a method’s usage, I’m immediately taken to the definition of the method. Inversely, holding the Command key and clicking on a method’s definition will either take me directly to its usage, or if it is used in multiple places, then it will show me each instance where the method is called, and take me to the one I select.

So note how we’re now in the run method, but somehow we’ve found ourselves in the yii/base/application class, even though we just clicked on the usage of an instance of craft/web/application. Well, this is as a result of how inheritance works in object-oriented programming. The craft/web/application class extends the yii/web/application class, which itself extends the yii/base/application class. Since the craft/web/application class does not contain a run method, the closest ancestor class, or superclass, that does contain a run method, in this case yii/base/application, is what is called. So we’re firmly in Yii land now, we’re allowing the framework to handle the running of this part of the request. There are two things of interest to us here, the first being that the handleRequest method is called on $this. Remember, however, that $this still represents an instance of craft/web/application, so when I want to navigate to the handleRequest method, I need to ensure that I either go to the method in the craft/web/application class, if one exists, otherwise to the closest superclass that contains one. Command-click takes me to an abstract method definition inside the yii/base/application class. An abstract method is a method signature, like a placeholder, that must be implemented by its subclasses. This icon here shows me which subclasses implement the handleRequest method. We, of course, are interested in craft/web/application, so that’s where we’ll head to next. 

One of the things that’s great about Craft, and Yii to an extent, is that stepping through and navigating the source code is so straightforward. There are magic methods used around the place, but not so many as to break the control flow or leave you guessing as to where to go to next, which is the case in many other frameworks. Also, the extensive use of DocBlocks that provide high-level descriptions of methods and their parameters, are wonderful to have. Here, for example, we can see that this method accepts two parameters, the request to be handled, as well as a boolean that specifies whether to skip the special case request handling stuff and go straight to the normal routing logic, which is false by default. Now, I wouldn’t be able to tell you what ​“special case request handling stuff” refers to without looking through the source code, but at least we get a sense of what that parameter is responsible for. The inline comments further help to illuminate the fact that special handling includes things such as processing resource requests – requests for static resources such as JavaScript files, CSS files or even images – telling bots not to index certain types of pages, setting basic security headers to help prevent XSS attack vectors in control panel pages, and so on.

If we inspect the usages of the handleRequest method then we can see where it is used with $skipSpecialHandling set to true. The PreviewController class is one example. It contains a preview action for live previewing entries and other elements, and it would make sense that when previewing an entry, we might not require so-called ​“special handling” of the request. 

I’m going to collapse this code block, as it’s over a hundred lines of code that performs various checks, leaving us with two other things of interest. Firstly, if we determine that this is an action request then we process it. Notice how the logic for doing this has been extracted into its own method. This is a refactoring technique that is commonly used for reducing cognitive load when reading source code, as the logic is abstracted away while the method name is nice and descriptive, but it can also be used when running the same logic from multiple places to keep our code DRY, as is the case here. Assuming this is a web request to the homepage, and therefore not an action request, we’re going to carry on to this line, in which the handleRequest method of this class’ parent class is called. So again, this works due to inheritance, by calling the closest ancestor class that contains a handleRequest method. Clicking on this takes us to the yii/web/application class which continues handling the request. What’s slightly unusual here, is that this instructs the request being handled, to resolve itself. It’s starting to sound like everyone is just passing the baton along, right? We have to be careful here, as clicking into the resolve method takes us into the yii/web/request class, when in fact we’re interested in the craft/web/request subclass, it’s just that PhpStorm doesn’t know this. Later in this video, we’ll use Xdebug to step through the code, which will help to eliminate this inconsistency.

Here, we get the result of the request by telling the URL manager to parse the request. This, in turn, calls the _getRequestRoute method, which checks whether there is a token in the URL, whether this is an element request, whether there is a URL route that matches the request, whether this is a ​“well-known” request, and finally, if none of the above match, we fetch the template route. Public template paths are paths that do not begin with an underscore, so $matches will evaluate to true, and the $path will evaluate to an empty string. And so this method will return an array of two values – the controller route and a set of parameters, which in this case will be the template with a value of an empty string, which will resolve to the index.twig template.

Ok, so I promised that we would go through this process again using Xdebug, which, used together with PhpStorm, is an incredible tool that I absolutely encourage you to start using if you don’t already. I’m using DDEV as my local development environment so all I need to do is run the ddev xdebug command, start listening for PHP debug connections, and now I can set some breakpoints in my code. Let me set one here, just so we can verify that the request does indeed go through the index.php file in the web folder. And I’ll set another one here so that we can see the result of parsing the request route.

Now, refreshing the page should prompt Xdebug to intercept the request at the first breakpoint that is encountered, and, as expected, we’re at the breakpoint that we set in the index.php file. Let’s resume execution, which brings us to the second breakpoint. Notice how the web browser is waiting for the response to come back. Xdebug has essentially paused runtime execution of the request, and in the debugger, we can see the stack trace leading up to this breakpoint, as well as the entire context, so all of the variables that are defined in the current scope, and we can inspect each and every one of them. 

Two very useful functions in Xdebug are ​“step over” and ​“step into”. ​“Step over” will step over the current line and onto the next line, whereas ​“step into” will step into any methods that are called in the current line. I’d like to see how the request route is parsed so let’s step into the method, and now I’ll step over this line of code. Since we skipped over this return statement, the condition must have evaluated to false. This time, however, we don’t skip over this return statement, meaning that the _getMatchedElementRoute function did not return false. So this gives us a more accurate representation of what is actually happening during the execution of this request, as opposed to what we assumed was happening when we stepped through the code manually. If we inspect the $route variable, we find the controller route of template/render, but this time the template parameter is set to index.twig and we have an additional parameter called variables, which contains the ​“Homepage” entry. So it’s not the case that visiting the homepage automatically renders the index template, as presupposed. Instead, the ​“Homepage” section, which I was very sneaky in failing to mention earlier, is set up to route homepage requests to the index.twig template, and that’s exactly what we’re seeing.

If we ​“step out” of this method, that is the opposite of stepping into a method, and out again, and again, and one final time, then we find ourselves back in the run method, however, now we have a $response that we can inspect. The response has a format of type ​“template”, but the content is still null, as there is one step remaining. So let’s set one more breakpoint, just after the send method is called on the response, and resume execution. Now the response content is populated with the raw HTML that will be sent back to the browser. We can resume execution and since we have no more breakpoints set, the request will complete.

Xdebug is incredibly useful for debugging and code exploration, and while you can get quite far using ​“dump-and-die”, it doesn’t give you anywhere as much control and insight into the lifecycle of requests. Let me give you another quick example.

Remember that one of the things that special request handling did was add some headers for us. Well, one of those headers is the X-Powered-By header, which Craft sets by default, but which you can disable using the sendPoweredByHeader general config setting. What if we wanted to test the effect of changing the value that is set, as a trivial example. We could set a breakpoint in the set method, which will break every time any header is set. But since we’re only interested in the X-Powered-By header, we can set a conditional breakpoint by right-clicking here and adding our condition, $name equals X-Powered-By. Let me just remove the other breakpoints I had set, and I should open the ​“Network” tab so we can see requests.

So here we are at our breakpoint, in which the X-Powered-By header is about to be set to ​“Craft CMS”. We can modify any variable, just like this, or we can even evaluate expressions, so let’s set $value to ​“CMS” replaced with ​“App”. And there we go – we’ve modified the state of the applicate during runtime execution. Back in my browser, I can inspect the response and there’s the X-Powered-By header set to ​“Cruft App”.

So if you’re still watching this video then congratulations, you’ve gotten an insight into the bootstrapping process of the Craft application and that puts you further ahead in the game than many other Craft developers. Perhaps you didn’t understand everything, and that’s ok, you’re already becoming more familiar with navigating the source code and I encourage you to practice doing so yourself as well. Reading source code is a skill that anyone can learn with practice and persistence.

Everything we’ve looked at until now was initialised by a web request. Craft also comes with console commands, which result in console requests and I’ll show you the entry point to those in just a moment. First, though, I mentioned that the .htaccess file is what is responsible for redirecting web requests to the index.php file, the entry point into the web application. That is the case if your site is hosted on an Apache web server, if your site runs on Nginx then there’ll be an Nginx configuration file with an equivalent redirect rule, albeit with a different syntax. 

And on the topic of web hosting, I’d like to share with you one of my recommendations when it comes to managed hosting, and that’s ArcusTech. This video has been made freely available to you thanks to financial support from ArcusTech, so bear with me while I highlight some of the benefits of using managed hosting for your Craft sites.

ArcusTech are well known in the Craft CMS space and they offer Craft optimised VPS plans. These are fully managed servers, in other words, you don’t have to know anything about Devops or server administration or have experts in-house, it’s almost like having ArcusTech provide your Devops expertise for you, handling things such as server updates, security patches and generally keeping everything running smoothly. One really nice feature is that Craft support engineers have direct access to your server if necessary, in case you need urgent support from Pixel & Tonic. Managed hosting plans start at $10 per month, which is incredibly affordable, and you can scale up or down any time, as needed. Where ArcusTech really shines is that their technical support is open 24 hours a day, 7 days a week, so you can email them or open a support request and get a prompt response from a real person. Data centres are available in the US and the EU, with Australia and other locations on the roadmap. So I encourage you to give arcustech​.com a look and if you prefer to outsource server administration and instead just build sites and do what you’re good at, then consider them for your next managed hosting project.

So back to our source code, I said that we would look at how console requests are initiated and that is through console commands, which go via the craft file, in the root of our project. This file is not too dissimilar from the index.php file, if you look closely. It’s also another file that has changed over time. As before, we load the shared bootstrap file, but instead of requiring the web bootstrap file, we require the console bootstrap file, which gives us an instance of the craft/console/application class, which we run, before exiting. 

One of the nice things here is that I can still use Xdebug, even though we’re going through the console. So I can set a breakpoint here and initiate a console request by running a console command. There are a few ways to do this – we can run ./craft to execute the file directly. The shebang at the top of the file tells the terminal to run the file using PHP. We can also run the file through PHP using php craft, but since I’m using DDEV I’ll use ddev craft to run it inside the Docker container. And there we are, runtime execution has been paused and we can see that the exit code is set to zero, to mean success.

So that’s it for this video, I hope you learned a thing or two and I’ll see you in the next one.