Commit Graph
127 Commits
Author SHA1 Message Date
J. Nick Koston a9731a7b26 Make the recorder LRU adjustment threadsafe (#88443) 2023-02-19 19:30:08 +01:00
J. Nick Koston 9df00bd285 Adjust recorder LRU cache to align with the number of entities (#88350)
* Adjust size of recorder LRU based on number of entities

If there are a large number of entities the cache would
get thrashed as there were more state attributes being
recorded than the size of the cache. This meant we had
to go back to the database to do lookups frequently when
an instance has more than 2048 entities that change
frequently

* add a test

* do not actually record 4096 states

* patch target

* patch target
2023-02-19 00:02:36 -05:00
J. Nick Koston 3ca9f3c0d5 Only build the recorder SupportedDialect enum once (#88402)
Every call to dialect_name was creating a new enum object
in the recorder
2023-02-18 20:21:14 -05:00
Erik Montnemery 83e5bf7ae8 Use entity_sources to determine integration in recorder platforms (#88382) 2023-02-18 07:21:41 -06:00
J. Nick Koston 289bab6f87 Make recorder platform attribute exclude integration aware (#88357) 2023-02-18 10:08:59 +01:00
epenet 185cd61cbd Add type hints to integration tests (recorder) (#88313) 2023-02-17 18:50:09 +01:00
Franck Nijhof ed79265843 Enable Ruff PT006 (#88165)
* Enable Ruff PT006

* Adjust existing cases

* Fix tests

* Remove unneeded parentheses
2023-02-15 14:09:50 +01:00
cbaf4764e7 Ensure MySQL tests cleanup connections and raise an exception if they do not (#87767)
Co-authored-by: epenet <6771947+epenet@users.noreply.github.com>
Co-authored-by: Erik <erik@montnemery.com>
2023-02-09 16:27:53 -06:00
Erik Montnemery 2fae82a9ef Really ensure recorder test fixture is setup before hass fixture (#87796) 2023-02-09 11:43:14 -06:00
J. Nick Koston c05a7b29e6 Terminate stale MySQL connections at the end of test runs (#87794) 2023-02-09 11:13:13 -06:00
J. Nick Koston 483b0cd017 Skip db locking test with postgresql/MySQL (#87760)
* Skip db locking test with postgresql/MySQL

https://github.com/home-assistant/core/pull/87756#issuecomment-1423828389

* switch to the guards instead
2023-02-09 13:58:32 +01:00
J. Nick Koston 77cdde94f3 Turn back off test_shutdown_before_startup_finishes test on MySQL (#87761) 2023-02-09 03:12:10 -06:00
Erik Montnemery 58618b2978 Improve recorder tests to use MariaDB / PostgreSQL (#87756) 2023-02-09 02:18:20 -06:00
epenet ba85fdcd61 Add type hints to integration tests (q-s) (#87706) 2023-02-08 08:12:54 -08:00
Erik MontnemeryandJ. Nick Koston 94519de8dd Upgrade SQLAlchemy to 2.0.2 (#86436)
Co-authored-by: J. Nick Koston <nick@koston.org>
2023-02-08 08:17:32 -06:00
720f51657d Add CI job which runs recorder tests on PostgreSQL (#80614)
Co-authored-by: Franck Nijhof <git@frenck.dev>
Co-authored-by: J. Nick Koston <nick@koston.org>
2023-02-07 09:46:14 -06:00
epenet 59ca7780fa Add typing to tests with single hass argument (#87631) 2023-02-07 15:01:16 +01:00
J. Nick Koston 389fc515a1 Prevent overly large attributes from being stored in the database (#87105) 2023-02-02 11:49:33 -06:00
J. Nick Koston 1fd58b6cb7 Fix a few cartesian products in recorder tests (#87106)
Fixes
```
recorder/test_init.py:251: SAWarning: SELECT statement has a cartesian product between FROM element(s) "states" and FROM element "state_attributes".  Apply join condition(s) between each element to resolve.
```
2023-02-02 07:47:09 +01:00
J. Nick Koston c612a92cfb Use python defaults for comparing State, LazyState, and Event objects (#86856)
* Speed up comparing State and Event objects

Use default python implementation for State and Event __hash__ and __eq__

The default implementation compared based on the id() of the object
which is effectively what we want here anyways. These overrides are
left over from the days when these used to be attrs objects

By avoiding implementing these ourselves all of the equality checks
can happen in native code

* tweak

* adjust tests

* write out some more

* fix test to not compare objects

* more test fixes

* more test fixes

* correct stats tests

* fix more tests

* fix more tests

* update sensor recorder tests
2023-01-29 13:31:43 -05:00
Erik Montnemery fea30c1ce9 Terminate strings at NUL when recording states and events (#86687) 2023-01-26 11:11:03 +01:00
epenet 30bf0634fe Add per-file-ignore to pylint (#86289) 2023-01-22 17:26:24 +01:00
Erik Montnemery b94e1e9ef8 Fire events when long term statistics is updated (#82492)
* Fire events when long term statistics is updated

* Allow the new events to be subscribed to by anyone

* Address review comments

* Finish renaming events

* Finish renaming events

* Fix do_adhoc_statistics

* Adjust tests

* Adjust tests
2022-11-24 22:01:36 +01:00
Erik MontnemeryandFranck Nijhof f4951a4f31 Add CI job which runs recorder tests on MariaDB (#80586)
Co-authored-by: Franck Nijhof <git@frenck.dev>
2022-10-19 13:04:28 +02:00
Erik Montnemery 31a787558f Ensure recorder test fixture is setup before hass fixture (#80528)
* Ensure recorder test fixture is setup before hass fixture

* Adjust more tests
2022-10-19 07:58:47 +02:00
Erik Montnemeryandepenet 2c206ad050 Fix flaky recorder test (#80246)
* Fix flaky recorder test

* Update tests/components/recorder/test_init.py

Co-authored-by: epenet <6771947+epenet@users.noreply.github.com>

Co-authored-by: epenet <6771947+epenet@users.noreply.github.com>
2022-10-14 09:10:38 +02:00
Erik Montnemery 04cc2ae264 Correct initialization of new databases (#80234) 2022-10-13 13:01:27 +02:00
Erik Montnemery 4462f2fc46 Fix recorder tests related to mysql (#80238) 2022-10-13 11:44:48 +02:00
Erik Montnemery 466c4656ca Refactor recorder migration (#80175)
* Refactor recorder migration

* Improve test coverage
2022-10-13 08:11:54 +02:00
Erik Montnemery 9aa6043255 Set character set to utf8mb4 when connecting to MySQL or MariaDB databases (#79755) 2022-10-11 14:01:46 +02:00
Erik Montnemery e8af007124 Disable echo for non SQLite databases (#80032)
* Disable echo for non SQLite databases

* Add test
2022-10-10 22:42:38 +03:00
Erik Montnemery 0a1fd36e03 Catch up with statistics after DB migration is done (#78469)
* Catch up with statistics after DB migration is done

* Don't access the database from the event loop

* Fix deadlocking test

* Fix test
2022-09-15 07:40:56 +02:00
Erik Montnemery fd6ffef52f Support non-live database migration (#72433)
* Support non-live database migration

* Tweak startup order, add test

* Address review comments

* Fix typo

* Clarify comment about promoting dependencies

* Tweak

* Fix merge mistake

* Fix some tests

* Fix additional test

* Fix additional test

* Adjust tests

* Improve test coverage
2022-07-22 15:11:34 +02:00
Erik Montnemery 606d544157 Use recorder get_instance function to improve typing (#75567) 2022-07-22 11:58:26 +02:00
Erik MontnemeryandJ. Nick Koston 5f2b4001f3 Separate recorder database schema from other classes (#72977)
* Separate recorder database schema from other classes

* fix logbook imports

* migrate new tests

* few more

* last one

* fix merge

Co-authored-by: J. Nick Koston <nick@koston.org>
2022-06-07 14:41:43 +02:00
J. Nick Koston f79e5e002b Ensure recorder shuts down when its startup future is canceled out from under it (#72866) 2022-06-01 22:13:09 -07:00
J. Nick KostonandPaulus Schoutsen 9c3f949165 Add live streaming logbook websocket endpoint (#72258)
Co-authored-by: Paulus Schoutsen <paulus@home-assistant.io>
2022-05-22 14:57:54 -05:00
J. Nick Koston 222baa53dd Make database access in the eventloop raise an exception (#71547) 2022-05-09 15:22:08 -05:00
J. Nick Koston a8aa0e1cca Add Estimated Database Size to the recorder system health (#71463) 2022-05-07 21:02:54 -07:00
J. Nick KostonandMartin Hjelmare eb77f8db85 Complete strict typing for recorder (#71274)
* Complete strict typing for recorder

* update tests

* Update tests/components/recorder/test_migrate.py

Co-authored-by: Martin Hjelmare <marhje52@gmail.com>

* Update tests/components/recorder/test_migrate.py

Co-authored-by: Martin Hjelmare <marhje52@gmail.com>

* Remove the asserts

* remove ignore comments

Co-authored-by: Martin Hjelmare <marhje52@gmail.com>
2022-05-04 12:22:50 -05:00
J. Nick Koston 13ce0a7d6a Fix history using pre v25 queries during v26 migration (#71295) 2022-05-04 08:56:50 -07:00
J. Nick Koston e30940ef2a Move processing of recorder service call arguments into services.py (#71260) 2022-05-03 13:56:22 -07:00
J. Nick Koston e1be6dd34f Move recorder services to services.py (#71249) 2022-05-03 10:38:44 -07:00
J. Nick Koston 29bda196b5 Break apart recorder into tasks and core modules (#71222) 2022-05-02 21:53:56 -07:00
J. Nick Koston 75026f9fed Separate recorder logic for state_changed and non-state_changed events (#71204) 2022-05-02 17:22:53 -05:00
J. Nick Koston 188040b8bb Use lambda_stmt for recorder queries and migrate them to queries module (#71219) 2022-05-02 15:17:21 -07:00
J. Nick KostonandPaulus Schoutsen 5db014666c Avoid recording state_changed events in the events table (#71165)
* squash

fix mypy

* Update homeassistant/components/recorder/models.py

Co-authored-by: Paulus Schoutsen <paulus@home-assistant.io>

* pass all columns

* fix commented out code

* reduce logbook query complexity

* merge

* comment

Co-authored-by: Paulus Schoutsen <paulus@home-assistant.io>
2022-05-02 02:10:34 -05:00
J. Nick Koston c23866e5e5 De-duplicate event data into a new event_data table (#71135) 2022-05-01 21:01:17 -05:00
J. Nick Koston 6ce768465f Improve purge performance for non-sqlite databases (#70576) 2022-04-26 13:11:57 -07:00
J. Nick Koston 1c4a785fb3 Prevent autoflush from happening during attrs lookup (#70768) 2022-04-26 13:04:58 -07:00