The Document AI of ComPDFKit Processor is a powerful tool for document format conversion, OCR, form recognition, layout analysis, and restoration. It enhances the quality of document processing, but it can consume significant server memory resources. However, the manual memory resource release by users is often challenging and unreliable.
To address this issue and optimize user experience, ComPDFKit Processor 1.3.0 introduces a new technology update: asynchronous task request and automatic memory release.
In this article, we will focus on the core technologies behind ComPDFKit Processor's asynchronous task request and memory release capabilities. We will explore the asynchronous queuing system implemented using RabbitMQ and the Docker Container Monitoring feature.
Key Technologies in Async Memory Release
The asynchronous request and container memory release in ComPDFKit Processor 1.3.0 rely on the following two core technologies:
1. Asynchronous Communication
ComPDFKit Processor utilizes RabbitMQ as its queuing framework for asynchronous task processing. RabbitMQ is a messaging broker - an intermediary for messaging. It gives your applications a common platform to send and receive messages, and your messages a safe place to live until received.
2. Docker Container Monitoring
To efficiently manage memory resources, ComPDFKit Processor employs Docker Container Monitoring. This technology continuously monitors the memory usage of containers. Docker Container Monitoring is the process of collecting metrics, traces, logs, and other observability data to improve the health and performance of containerized applications.
ComPDFKit Processor: Asynchronous Task Request Flow
The asynchronous task request flow in ComPDFKit Processor follows a well-defined process to ensure efficient task execution.
1. User Request: Users initiate a task by uploading files and specifying processing parameters. The system generates a unique task ID but does not immediately start processing the task.
2. Task Queuing: All tasks are entered into a queue, ensuring fair and orderly processing.
3. Container Assignment: The queue assigns tasks to idle containers for processing. This distribution of tasks optimizes resource allocation and promotes efficient task execution.
4. Processing Started: Tasks change their status to "Processing Started" once assigned to a container.
5. Result Retrieval: Users can request the archived result of their task and receive it promptly.
Asynchronous Memory Release Process
ComPDFKit Processor's memory release process operates seamlessly alongside task execution, ensuring optimal resource utilization.
- Container Memory Check: As a task enters a processing container, the container performs a memory check. If the memory usage exceeds the predefined threshold, the task is returned to the queue for reassignment to an idle container. This mechanism guarantees smooth task execution without disrupting ongoing processes.
- Scheduled Memory Release: ComPDFKit Processor's monitoring service periodically releases memory for containers that exceed the memory threshold. However, ongoing tasks are exempt from memory release to prevent interruptions.
Conclusion
With this update, ComPDFKit Processor avoids improper resource usage, minimizing waste and maximizing performance. Contact ComPDFKit team or the Web page to learn more about the ComPDFKit Processor.
