php - How to store/access data in mariadb which is updated every 15 seconds

I'd like to download some json-data (which gets updated every 15 seconds) and store it in my maria-db with a PHP-Script. Unfortunately, the database-update queries take between 1 second and sometimes up to 60 seconds, depending on the json-data-size. So sometimes I'm dead-locking myself with the write-queries who take longer than 15 seconds and as soon as I read/process the data I'm blocking all the write-queries as well.

Obviously, I do have the wrong approach and it's more complicated than I thought. Does anyone have a good idea how such a job can be done professionally, with a continuous update possibility and not blocking the updates itself when I read the data?

Thanks for any hints!

PS: Currently I'm using an InnoDB-Table, and to speed up the inserts I've set the auto-commit to 0 and update everything in a transaction. I had the fastest results with LOCK TABLES for WRITE, but of course this blocks the read access as well.

Solution:

Simply updating some data into MariaDB shouldn't take that long unless the update you're doing is complex. What you could consider is inserting the raw JSON (maybe even in a document database instead) and have a background process triggered by a cronjob to read from the stored raw JSON to update MariaDB.

Additionally you could consider inserting data rather than updating. This will prevent deadlocks from happening. Doing so might require you to change your data model, so it might not be the solution you're looking for.

Other than the above I'd recommend you look into the process you've setup and split it into multiple steps which can be run individually. Doing so allows you fine-grained control over the timing and triggers for each step, which will prevent deadlocks if setup properly.