- Updated unit test cases pipeline
- Added support to override uploaded blob access tier
- Updated unit test cases pipeline
- Added support to upload test cases results on Azure blob
- Message Lock Renewal: Implemented a mechanism to automatically renew message locks during processing. This ensures that messages remain active and are not returned to the queue for reprocessing while they are being handled.
- Concurrent Message Processing: Enhanced the system to process messages concurrently using a number of worker threads equal to the number of available CPU cores by default. Users can override this default by specifying the
max_concurrent_messagesparameter, for example,core.get_topic(topic_name=topic_name, max_concurrent_messages=10). This optimization leverages system resources for improved performance and throughput. - Completion Acknowledgement: Updated the processing flow to wait until message processing is fully completed before sending the acknowledgement of message completion. This change ensures reliable processing and accurate message handling.
- Version Tracking: Introduced a
version.pyfile to maintain and track the package version. This addition facilitates version control and package management. - Unit Test Updates: Updated unit test cases to cover the new features and enhancements, ensuring robust testing and quality assurance.
- Documentation Update: Updated the README file to reflect the new features and enhancements, providing clearer guidance and information for users.
- Adds extra checks in service bus for retries
- Additional logging done if there is no network and service bus crashes
- Service bus subscription picks up messages even if there is network outtage
- Added Unit Test Cases
- Updated Readme file
- Removed extra logs
This fixes the inconsistent listening behavior for the subscription of a topic from core.
- Due to the usage of
for message in receiverlogic in the topic, it is unknown when the loop will end and the core will stop listening to the topic. - This is tried with various combinations and figured out that at probably 4 hours from launch, this happens. The root cause for this is the socket / connection timeout of the underlying amqp client which throws an exception when trying to iterate next message
Fix made:
- The above for loop is kept in an infinite while loop which triggers the creation of the receiver and subsequent listening to the messages.
- This was tested overnight with messages varying in the time differences (1h to 3h)
- Specific method to look for the fix
start_listening
Reference task: 302
- Removed validation from QueueMassage Class
- Added name_starts_with parameter in Storage
- Added Local Storage
- Added TopicAbstract and TopicConfig to package
- Adding files to python package
- Authorization methods included
- Simulated and Hosted Authorizers implemented
- Examples written for both
- Readme updated appropriately.
- Fixed configuration issues
- Fixed all the review comments
- Refactor code
- Removed unused code
- Added local dev env using TDEI-local-server Package
- Added local env support to -
- Logger
- Queue
- Topic
- Storage
- Added get_remote_url function in FileEntity class which will return the uploaded file url
- Fixed Storage Blob stream
- Fixed threading
- Removed blocking call from topic subscription
- Added TODOs and FIXMEs
- Added filtered exception handling
- Removed unused packages
- Fixed Topic subscription
- Introduces Topic and subscription methods in Core.
- Added methods
- Core.ge_topic()
- Added classes and methods
- Topic
- publish
- subscribe
- Topic