Coldfusion 10/2016/2018: Services are not starting up with lots of cfclasses files

You may be in the situation that ColdFusion services are not starting up and a manual startup yields no useful information, too. It simply failed after it tries to startup the services for a very long time. If services are starting up it may takes an extreme long time to start.

Start-Service : Failed to start service 'ColdFusion 10 Application Server cfusion (ColdFusion 10 Application Server cfusion)'.
At line:1 char:1
+ Start-Service Cold*
+ ~~~~~~~~~~~~~~~~~~~
   + CategoryInfo  : OpenError: (System.ServiceProcess.ServiceController:ServiceController) [Start-Service],
  ServiceCommandException
   + FullyQualifiedErrorId : StartServiceFailed,Microsoft.PowerShell.Commands.StartServiceCommand

Background:

There are for sure more possible situation that may cause a ColdFusion startup failure, but if you have not changed anything and only rebooted/restarted your server/coldfusion after some OS updates and ColdFusion is now failing to start without having anything changed in ColdFusion you should check this issue very first. From my point of view this issue here is the most common issue that everyone will experience sooner or later.

The fact is that your cfclasses cache of ColdFusion may has been filled with an extreme large number of class files. Check how many files the folder C:\ColdFusionXX\cfusion\wwwroot\WEB-INF\cfclasses contains. If you issue the dir command in this folder you may see ~150.000 to 200.000 class files inside. Than your are for sure affected! No idea why Adobe ColdFusion is not cleaning up this folder from time to time. This cfclasses folder can fills up your disk slowly, too. It holds code for all your CFML templates loaded over the course of ColdFusion server lifetime - including the 1000 times replaced CFML code if you had a lot of CFML changes that gone live.

The files are there for only one reason - service startup performance. ColdFusion and your Applications can start up a lot faster if the java class files are already on disk. If the files do not exist ColdFusion need to compile all your CFML templates first into java byte code before the first page is delivered to the world. This CFML to Java byte code compiling will cause for sure 100% CPU on your server for an extended time (e.g. 3-5 minutes) if you ever wondered what ColdFusion is doing there with your CPU. How long this CPU flat line takes highly depends on how many CFML templates your application has and how much load you have.

After asking support they identified the root cause and found that Tomcat scans C:\ColdFusionXX\cfusion\wwwroot\WEB-INF and all its sub-directories on startup for *.tld files. The hotfix CF-4199617 has disabled this TLD scanning as it is not needed. This feature is named TLD scanning (Discovery of tag libraries). The problem here is simple - a file system directory walk that is scanning hundred thousands of files takes very long time or tooo long before the service startup timeout is reached.

Solutions:

  • Recommended: Install Adobe hotfix for CF-4199617: ColdFusion service does not start with large cfclasses folder. You may need to contact support to get the updated jasper.jar file. We have tested with up to 1.000.000 class files and the coldfusion service starts within 30 seconds! Without the hotfix you will be unable to start the CF service automatically and manually. Service start will always run into a timeout.
    • Make sure you are adding a scheduled task to your operating system that cleans the folder. A powershell script that deletes all files older than one day is a very handy way.
  • Workaround 1: Delete all class files or if you are in a hurry - rename C:\ColdFusionXX\cfusion\wwwroot\WEB-INF\cfclasses to C:\ColdFusionXX\cfusion\wwwroot\WEB-INF\cfclasses.old and delete the folder later. The folder and files will be recreated automatically on service start.
    • Make sure you are not running into this issue again by adding a scheduled task to your operating system that cleans the folder. A powershell script that deletes all files older than one day is a very handy way.
  • Workaround 2: Disable cfclasses cache in CF Administrator. This is not the default and not recommended as it causes a lot of CPU load after a server start. But just as a note - there have been bugs in the past when this was absolutely required to workaround some java bugs. As always - the class files will stay in memory of ColdFusion server as long as you do not restart your server.

History:

  • 03/09/2019: Per Adobe the jasper fix may be included in CF2016 Updater 12 / CF2018 Updater 5 and later.
  • 20/11/2017: Received a hotfix (cfusion\runtime\lib\jasper.jar) for CF2016. May be included in CF2016 Updater 6 and later. CF-4199617: ColdFusion service does not start with large cfclasses folder.
  • 01/12/2015: Documented workaround for CF10, but this issue exists for many more years and older versions.
Rating
Average: 7.8 (8 votes)