Commit Graph
64 Commits
Author SHA1 Message Date
Franck Nijhof ed79265843 Enable Ruff PT006 (#88165)
* Enable Ruff PT006

* Adjust existing cases

* Fix tests

* Remove unneeded parentheses
2023-02-15 14:09:50 +01:00
J. Nick Koston db503c0daa Always use UTC time for the MariaDB/MySQL session (#87583) 2023-02-09 11:39:45 -06:00
epenet ba85fdcd61 Add type hints to integration tests (q-s) (#87706) 2023-02-08 08:12:54 -08:00
720f51657d Add CI job which runs recorder tests on PostgreSQL (#80614)
Co-authored-by: Franck Nijhof <git@frenck.dev>
Co-authored-by: J. Nick Koston <nick@koston.org>
2023-02-07 09:46:14 -06:00
epenet 4142f0d15d Add return type to tests without arguments (#87613)
* Add return type to tests without arguments

* Black

* Cancel fixture amends
2023-02-07 14:20:06 +01:00
J. Nick Koston c11f3ffa17 Speed up purge time with newer MariaDB versions (#87409)
* Speed up purge time with newer MariaDB versions

* fix

* document

* document

* document

* rename

* self review

* Update homeassistant/components/recorder/util.py

* fixes
2023-02-05 17:58:34 -05:00
J. Nick Koston 5284837c8f Add a repair issue when using MariaDB is affected by MDEV-25020 (#87040)
closes https://github.com/home-assistant/core/issues/83787
2023-01-31 20:42:07 +01:00
Franck Nijhof a79885ceaf Enable Ruff SIM117 (#86783) 2023-01-27 11:52:49 +01:00
Joakim Plate c576a68d33 Upgrade pytest-aiohttp (#82475)
* Upgrade pytest-aiohttp

* Make sure executors, tasks and timers are closed

Some test will trigger warnings on garbage collect, these warnings
spills over into next test.

Some test trigger tasks that raise errors on shutdown, these spill
over into next test.

This is to mimic older pytest-aiohttp and it's behaviour on test
cleanup.

Discussions on similar changes for pytest-aiohttp are here:
https://github.com/pytest-dev/pytest-asyncio/pull/309

* Replace loop with event_loop

* Make sure time is frozen for tests

* Make sure the ConditionType is not async

  /home-assistant/homeassistant/helpers/template.py:2082: RuntimeWarning: coroutine 'AsyncMockMixin._execute_mock_call' was never awaited
    def wrapper(*args, **kwargs):
  Enable tracemalloc to get traceback where the object was allocated.
  See https://docs.pytest.org/en/stable/how-to/capture-warnings.html#resource-warnings for more info.

* Increase litejet press tests with a factor 10

The times are simulated anyway, and we can't stop the normal
event from occuring.

* Use async handlers for aiohttp

tests/components/motioneye/test_camera.py::test_get_still_image_from_camera
tests/components/motioneye/test_camera.py::test_get_still_image_from_camera
tests/components/motioneye/test_camera.py::test_get_stream_from_camera
tests/components/motioneye/test_camera.py::test_get_stream_from_camera
tests/components/motioneye/test_camera.py::test_camera_option_stream_url_template
tests/components/motioneye/test_camera.py::test_camera_option_stream_url_template
  /Users/joakim/src/hass/home-assistant/venv/lib/python3.9/site-packages/aiohttp/web_urldispatcher.py:189: DeprecationWarning: Bare functions are deprecated, use async ones
    warnings.warn(

* Switch to freezegun in modbus tests

The tests allowed clock to tick in between steps

* Make sure skybell object are fully mocked

Old tests would trigger attempts to post to could services:

```
DEBUG:aioskybell:HTTP post https://cloud.myskybell.com/api/v3/login/ Request with headers: {'content-type': 'application/json', 'accept': '*/*', 'x-skybell-app-id': 'd2b542c7-a7e4-4e1e-b77d-2b76911c7c46', 'x-skybell-client-id': '1f36a3c0-6dee-4997-a6db-4e1c67338e57'}
```

* Fix sorting that broke after rebase
2022-11-29 22:36:36 +01:00
Erik Montnemery 2fe8e95309 Add helper to calculate statistic period start and end (#82493)
* Add helper to calculate statistic period start and end

* Don't parse values in resolve_period

* Add specific test for resolve_period

* Improve typing

* Move to recorder/util.py

* Extract period schema
2022-11-26 13:00:40 -05:00
Erik MontnemeryandFranck Nijhof f4951a4f31 Add CI job which runs recorder tests on MariaDB (#80586)
Co-authored-by: Franck Nijhof <git@frenck.dev>
2022-10-19 13:04:28 +02:00
Erik Montnemery 31a787558f Ensure recorder test fixture is setup before hass fixture (#80528)
* Ensure recorder test fixture is setup before hass fixture

* Adjust more tests
2022-10-19 07:58:47 +02:00
epenet 019d297ff0 Apply hass-relative-import to recorder tests (#78734) 2022-09-19 12:35:03 +02:00
Erik Montnemery 606d544157 Use recorder get_instance function to improve typing (#75567) 2022-07-22 11:58:26 +02:00
J. Nick Koston 00a79635c1 Revert "Remove sqlalchemy lambda_stmt usage from history, logbook, and statistics (#73191)" (#73917) 2022-06-23 13:59:55 -05:00
J. Nick Koston 7ae8bd5137 Remove sqlalchemy lambda_stmt usage from history, logbook, and statistics (#73191) 2022-06-07 16:15:50 -07:00
Erik MontnemeryandJ. Nick Koston 5f2b4001f3 Separate recorder database schema from other classes (#72977)
* Separate recorder database schema from other classes

* fix logbook imports

* migrate new tests

* few more

* last one

* fix merge

Co-authored-by: J. Nick Koston <nick@koston.org>
2022-06-07 14:41:43 +02:00
Erik Montnemery edd7a3427c Remove support for databases without ROW_NUMBER (#72092) 2022-05-18 21:52:38 -05:00
Erik Montnemery 037f6947d8 Fail recorder setup with unsupported dialect or version (#70888) 2022-05-18 16:52:46 +02:00
J. Nick Koston a4c1bcefb9 Tune sqlite based on configured settings (#72016) 2022-05-17 18:12:15 -05:00
J. Nick KostonandPaulus Schoutsen 98809675ff Convert history queries to use lambda_stmt (#71870)
Co-authored-by: Paulus Schoutsen <paulus@home-assistant.io>
2022-05-15 10:47:29 -05:00
J. Nick Koston a8f1dda004 Use ciso8601 for parsing MySQLdb datetimes (#71818)
* Use ciso8601 for parsing MySQLDB datetimes

The default parser is this:

https://github.com/PyMySQL/mysqlclient/blob/5340191feb16b1f99a7b43fe7c74a2f690138cb1/MySQLdb/times.py#L66

* tweak

* tweak

* add coverage for building the MySQLdb connect conv param
2022-05-13 19:26:09 -05:00
J. Nick Koston 222baa53dd Make database access in the eventloop raise an exception (#71547) 2022-05-09 15:22:08 -05:00
J. Nick Koston 29bda196b5 Break apart recorder into tasks and core modules (#71222) 2022-05-02 21:53:56 -07:00
J. Nick KostonandErik Montnemery f073f17040 Refactor tracking of the recorder run history (#70456)
Co-authored-by: Erik Montnemery <erik@montnemery.com>
2022-04-26 09:59:43 -10:00
Erik Montnemery d045e8678d Use recorder test fixtures in tests (#70652) 2022-04-25 14:23:52 +02:00
Erik Montnemery eb3458a3d2 Add MutexPool for recorder tests (#69410)
* Add MutexPool for recorder tests

* Fix get_schema_version

* Update test test_last_run_was_recently_clean

* Update test test_shutdown_before_startup_finishes

* Revert comments in test_write_lock_db

* Make the MutexPool lock a class variable

* Remove stale comment

* Move MutexPool

* Tweak debug prints
2022-04-12 07:41:46 -10:00
J. Nick KostonandMartin Hjelmare 72fffde77a Fix spelling of periodic in recorder (#69658)
Co-authored-by: Martin Hjelmare <marhje52@gmail.com>
2022-04-07 23:37:02 -10:00
Erik Montnemery 4a7c978f69 Speedup recorder test test_write_lock_db (#69578) 2022-04-07 15:36:58 +02:00
J. Nick Koston bde3646293 Always use a commit interval of 0 for the in memory db in tests (#69330) 2022-04-05 12:06:06 -10:00
J. Nick Koston ec131d685e Auto repack the database on the second sunday of the month (#69314) 2022-04-04 21:39:12 -10:00
bc862e97ed Use a dedicated executor pool for database operations (#68105)
Co-authored-by: Erik Montnemery <erik@montnemery.com>
Co-authored-by: Franck Nijhof <git@frenck.dev>
2022-03-17 23:09:01 -10:00
Stefan Agner 0a9927d18e Avoid locking the database for non-SQLite backends (#63847)
* Avoid locking the database for non-SQLite backends

Currently we only have a lock implementation for SQLite. Just return
success for all other databases as they are not expected to store data
in the config directory and the caller can assume that a backup can
be safely taken.

This fixes `RuntimeError: generator didn't yield` errors when creating
a backup with the current Supervisor dev builds.
2022-01-11 16:17:56 +01:00
f0006b92be Allow to lock SQLite database during backup (#60874)
* Allow to set CONF_DB_URL

This is useful for test which need a custom DB path.

* Introduce write_lock_db helper to lock SQLite database

* Introduce Websocket API which allows to lock database during backup

* Fix isort

* Avoid mutable default arguments

* Address pylint issues

* Avoid holding executor thread

* Set unlock event in case timeout occures

This makes sure the database is left unlocked even in case of a race
condition.

* Add more unit tests

* Address new pylint errors

* Lower timeout to speedup tests

* Introduce queue overflow test

* Unlock database if necessary

This makes sure that the test runs through in case locking actually
succeeds (and the test fails).

* Make DB_LOCK_TIMEOUT a global

There is no good reason for this to be an argument. The recorder needs
to pick a sensible value.

* Add Websocket Timeout test

* Test lock_database() return

* Update homeassistant/components/recorder/__init__.py

Co-authored-by: Erik Montnemery <erik@montnemery.com>

* Fix format

Co-authored-by: J. Nick Koston <nick@koston.org>
Co-authored-by: Erik Montnemery <erik@montnemery.com>
2021-12-07 13:16:24 +01:00
Erik Montnemery be4e9f91b6 Change minimum supported SQLite version to 3.31.0 (#59073) 2021-11-04 16:34:35 +01:00
Erik Montnemery dfa50a842a Simplify recorder PgSQL version checks (#58533) 2021-10-27 16:05:40 +02:00
Erik Montnemery e9ba5f3b4b Warn when recorder connects to an unsupported database (#58161) 2021-10-26 13:41:59 +02:00
Erik Montnemery 8ef8838801 Correct detection of row_number support for MariaDB (#57663) 2021-10-14 11:19:39 -07:00
Erik Montnemery 0139bfa749 Detect if mysql and sqlite support row_number (#57475) 2021-10-11 21:17:18 -07:00
Erik MontnemeryandPaulus Schoutsen 5976f898da Add WS API for removing statistics for a list of statistic_ids (#55078)
* Add WS API for removing statistics for a list of statistic_ids

* Refactor according to code review, enable foreign keys support for sqlite

* Adjust tests

* Move clear_statistics WS API to recorder

* Adjust tests after rebase

* Update docstring

* Update homeassistant/components/recorder/websocket_api.py

Co-authored-by: Paulus Schoutsen <paulus@home-assistant.io>

* Adjust tests after rebase

Co-authored-by: Paulus Schoutsen <paulus@home-assistant.io>
2021-09-27 23:30:13 +02:00
Paulus Schoutsen 557cc792e9 Fix SQLAlchemy test warnings (#54116) 2021-08-08 20:33:47 -07:00
Erik Montnemery e16a8063a5 Compile statistics for energy sensors (#50829)
* Compile statistics for energy sensors

* Update tests

* Rename abs_value to state

* Tweak

* Recreate statistics table

* Pylint

* Try to fix test

* Fix statistics for multiple energy sensors

* Fix energy statistics when last_reset is not set
2021-05-20 13:05:15 +02:00
J. Nick Koston e7f7e61e88 Ensure a wal checkpoint is scheduled nightly (#50746) 2021-05-17 16:27:51 -05:00
J. Nick Koston 72288710ca Increase the sqlite cache size from ~2MiB to 8MiB (#50747) 2021-05-17 19:42:12 +02:00
J. Nick Koston c10836fcee Upgrade to sqlalchemy 1.4.11 (#49538) 2021-04-21 20:29:36 -10:00
J. Nick KostonandMartin Hjelmare 6d137d2316 Increase recorder test coverage (#49362)
Co-authored-by: Martin Hjelmare <marhje52@gmail.com>
2021-04-19 08:22:38 -07:00
J. Nick KostonandPaulus Schoutsen 9368891b1b Live db migrations and recovery (#49036)
Co-authored-by: Paulus Schoutsen <paulus@home-assistant.io>
2021-04-11 20:43:54 -10:00
Franck NijhofandMartin Hjelmare 08870690a6 Fix a collection of tests with missing asserts (#48127)
Co-authored-by: Martin Hjelmare <marhje52@gmail.com>
2021-03-20 17:23:55 +01:00
J. Nick Koston e2fd255a96 Cleanup recorder tests (#46836) 2021-02-21 09:52:41 +01:00
J. Nick Koston 9b69549f73 Recover and restart the recorder if the sqlite database encounters corruption while running (#46612) 2021-02-19 21:26:24 -10:00