Get the solution ↓↓↓
I'd like to download some json-data (which gets updated every 15 seconds) and store it in my maria-db with a PHP-Script. Unfortunately, the database-update queries take between 1 second and sometimes up to 60 seconds, depending on the json-data-size. So sometimes I'm dead-locking myself with the write-queries who take longer than 15 seconds and as soon as I read/process the data I'm blocking all the write-queries as well.
Obviously, I do have the wrong approach and it's more complicated than I thought. Does anyone have a good idea how such a job can be done professionally, with a continuous update possibility and not blocking the updates itself when I read the data?
Thanks for any hints!
PS: Currently I'm using an InnoDB-Table, and to speed up the inserts I've set the auto-commit to 0 and update everything in a transaction. I had the fastest results with LOCK TABLES for WRITE, but of course this blocks the read access as well.
Simply updating some data into MariaDB shouldn't take that long unless the update you're doing is complex. What you could consider is inserting the raw JSON (maybe even in a document database instead) and have a background process triggered by a cronjob to read from the stored raw JSON to update MariaDB.
Additionally you could consider inserting data rather than updating. This will prevent deadlocks from happening. Doing so might require you to change your data model, so it might not be the solution you're looking for.
Other than the above I'd recommend you look into the process you've setup and split it into multiple steps which can be run individually. Doing so allows you fine-grained control over the timing and triggers for each step, which will prevent deadlocks if setup properly.
Our community is visited by hundreds of web development professionals every day. Ask your question and get a quick answer for free.
Find the answer in similar questions on our website.
Do you know the answer to this question? Write a quick response to it. With your help, we will make our community stronger.
PHP (from the English Hypertext Preprocessor - hypertext preprocessor) is a scripting programming language for developing web applications. Supported by most hosting providers, it is one of the most popular tools for creating dynamic websites.
The PHP scripting language has gained wide popularity due to its processing speed, simplicity, cross-platform, functionality and distribution of source codes under its own license.
https://www.php.net/
Welcome to the Q&A site for web developers. Here you can ask a question about the problem you are facing and get answers from other experts. We have created a user-friendly interface so that you can quickly and free of charge ask a question about a web programming problem. We also invite other experts to join our community and help other members who ask questions. In addition, you can use our search for questions with a solution.
Ask about the real problem you are facing. Describe in detail what you are doing and what you want to achieve.
Our goal is to create a strong community in which everyone will support each other. If you find a question and know the answer to it, help others with your knowledge.
