Highly available digital asset management can be important when there are thousands of people using it. Not everyone will be accessing it at the same time, but if even 100 users are stopped from getting their current task completed when it isn't available it can equate significant costs for a company.
DBGallery supports a high availibility (HA) environment primarily by being a stateless web application. Other aspects like retries when there is a lost database connection also help.
We see high availability in three distinct areas of system architecture:
Each of these are covered on the following pages.
In all cases, as part of best practices, each of these are strengthened when using multi-region or multi-datacenter configurations when availability is an absolute requirement. That decision may be in part based on current experience with other HA systems running on one or across multiple datacenters, as well as cost and budget.
A load balancer in front of multiple IIS web servers is used to ensure availability of the web application. Just two servers behind the load balancer would suffice for the purpose of availability. For high traffic systems, additional IIS servers may be added to spread the traffic load. DBGallery's web tier is stateless and so multiple copies of DBGallery’s web application may be combined with any network or application load balancing solution.
Database replication and auto-switching of primary database sources is the standard means to achieve high availability at the database level. There is nothing required within DBGallery itself to support this. When the connection is switched to the new database, DBGallery will continue to run, retrying a query if a connection was lost during the switch. The possible side-effects of this is that current operations may be lost: a search may be interrupted, or a download may need to be retried, as examples.
While there are numerous sources for database replication best practices and configurations, one place to start would be Amazon’s overview for RDS high availability: https://aws.amazon.com/rds/ha/
DBGallery is a robust application. The web application is stateless and recoverability depends solely on the integrity and availability of the file system and MySQL database. DBGallery will survive temporary lost connections to either of those. Search and thumbs previews continue to work with the file system disconnected, although full size image previews are not. Also helping with avoiding downtime, there is no regular required maintenance which requires downtime.
While DBGallery will not have downtime when it is upgraded from time to time, current user operations may be interrupted. These include searches, downloads and uploads. Therefore it is best to implement updates during off hours, or when that is not possible, notify users of scheduled update times, warning of possible interruptions.
DBGallery can be integrated with Azure Application Insights for detailed analysis and alerting. We incorporate Azure Application Insights internally as part of monitoring. It can be integrated with other similar platforms on demand.
Our log files can be sent to monitoring tools such as Splunk and Datadog.
The application uses log4net and additional log4net connectors may be added for integration with 3rd party services.
Because DBGallery is a stateless web application with a standard relational database backend, high availability must be achieved on the infrastructure level and not in the DBGallery itself. Therefore server administration and database administration expertise are required. Each client’s datacenter environment will have their own means to configure HA systems. Each will have nuances which will make some areas easier to configure and be more robust, while some areas may be weaker. Cost is also a factor in achieving high availability, especially when considering cross-region and cross-datacenter options. This means budget can be a factor in the level of high availability achieved.
Related links for further reading on high availability: