Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Ampache API improvements #1078

Merged
merged 83 commits into from
Oct 5, 2023
Merged

Conversation

paulijar
Copy link
Collaborator

@paulijar paulijar commented Jul 27, 2023

The purpose of this PR is to:

  • Provide support for Ampache API versions 4, 5, and 6, depending on the version requested on handshake
  • Backfill some missing API features from various API versions. Still, not everything from the API specification will be supported. Priority is on features which are a good match to our existing SW infrastructure and/or are already used by some client.

The scope of this PR ended up being larger than initially expected. Here's the final change log:

Added

  • New methods:
    • rate
    • get_similar
    • genres, genre, genre_artists, genre_albums, genre_songs
    • bookmarks, get_bookmark, bookmark_create, bookmark_edit, bookmark_delete
    • live_streams, live_stream, live_stream_create, live_stream_edit, live_stream_delete
    • list
    • browse
    • user_preference and user_preferences with mock-up content
    • advanced_search with partial support, not all search rules supported and some operators work only with MySQL/MariaDB
  • Support for the type album_artist in the method get_indexes
  • Support for the parameter album_artist in the method artists
  • Support for the type playlist in the method stats
  • Support for the type playlist in the methods downlaod and stream
  • Support for the type playlist in the method flag
  • Support for the parameter top50 in the method artist_songs
  • Supoort for the filter highest in the method stats
  • Support for the parameter include in the methods album, albums, artist, and artists
  • Fields time, albumcount, songcount, prefix, and basename to the artist type results
  • Fields time, diskcount, songcount, prefix, and basename to the album type results
  • Fields disk, format, stream_format, stream_bitrate, stream_mime, and playlisttrack to song type results
  • Fields time, size, bitrate, stream_bitrate, rating, and preciserating to podcast_episode type results
  • Fields rating and preciserating to podcast type results
  • Fields flag, rating and preciserating to playlist type results
  • Null-valued fields language, lyrics, mode, rate, replaygain_album_gain, replaygain_album_peak, replaygain_track_gain, replaygain_track_peak, r128_album_gain, and r128_track_gain to song type results
  • In JSON-mode only, field artists to song and album type results
  • All the fields of handshake response on the response of ping within a valid session

Changed

  • Follow the APIv5 conventions if version 5.x.x requested by the client on handshake
  • Follow the APIv6 conventions if version 6.0.0 or higher requested by the client on handshake
  • Follow the APIv6 conventions if the client doesn't specify any version
    • this may be overridden using the config.php key music.ampache_api_default_ver
  • The URLs returned in the art tag of the entities are now cache-friendly, i.e. don't depend on the session
  • Terminate all related sessions immediately when API key deleted; previously, this happened upon session timeout
  • Fields rating and preciserating now show the user-given rating instead of constant 0 on all applicabale result objects

@paulijar paulijar self-assigned this Jul 27, 2023
@paulijar paulijar mentioned this pull request Jul 27, 2023
@paulijar paulijar force-pushed the feature/909_Ampache_API_improvements branch from e165a8c to a8b3f62 Compare August 12, 2023 17:00
@paulijar paulijar force-pushed the feature/909_Ampache_API_improvements branch 2 times, most recently from 5a3d3f5 to 673c0e5 Compare September 3, 2023 10:10
@paulijar paulijar force-pushed the feature/909_Ampache_API_improvements branch from a5e6a62 to 1b61a33 Compare September 3, 2023 14:58
@paulijar paulijar force-pushed the feature/909_Ampache_API_improvements branch from b752ccf to 8b1a0c0 Compare September 10, 2023 15:08
@paulijar paulijar force-pushed the feature/909_Ampache_API_improvements branch from 8b1a0c0 to 1efcaa5 Compare September 20, 2023 20:58
The Last.fm connection must be set up on the server or the method will return
only empty results.
- Most AmpacheAPI methods now return only an array which is then passed to the
  `ampacheResponse()` method commonly within the `dispatch()` method. This
  saves us a lot of redundant calls to `ampacheResponse()`.

- Added type declarations to all method parameters and return values
…ndexes`

The support is only partial as the optional arguments `filter`, `add`, and
`update` may not be used together with this `type` value.
…bjects

All the returned `artist` objects now contain the fields `time`, `albumcount`,
and `songcount`. The returned `album` objects contain the new fields `time`
and `songcount`. These had been added to the API spec in the version 4.4.0.
To match the API spec v4.4.0.
…sponse

Also this has changed on the API spec v4.4.0. Obviously, the `handshake`
fields are included on the `ping` response only in case the client has
passed a valid session `auth` token as an argument.
For now, the API version information is not actually used for anything.
Got rid of the AmpacheUser utility class. This was used to share the
information about the authorized user between AmpacheMiddleware and
AmpacheController. Now we use the AmpacheSession for the same purpose.
This has the added benefit of sharing the current token, too, and we no
longer need to pass it around as an argument of dozens of functions.
The AmpacheSession also carries the information about the requested API
version which will come in handy very soon.
Move all the logic related to session management from AmpacheController
to AmpacheMiddleware. This achieves better separation of concerns. The
middleware was already previously handling part of the session
management logic.
…s >4

In the APIv4, most JSON responses had an anonymous array as the root
element. This has changed in APIv5 so that the root is now an object
with a single named property and array as its value. This is more
straight forward for us, and similar to how the Subsonic API has always
worked.
Added actions  `genres`, `genre`, `genre_artists`, `genre_albums`,
`genre_songs`. These are APIv5 replacements for the former "tag"
functions. We now provide then regardless of the API version. They are
otherwise identical to the "tag" functions but the the result elements
have the name "genre" instead of "tag".

Also, the `handshake` result as well as any results containing songs,
albums, or artists now use the key "genre" instead of "tag" when the
active API version is 5 or greater.
The new error format is used when the requested API version in the
session is greater than 4. The new format includes also a new property
"errorType" which is now set as "system" on all errors.
For some reason, the Kodi Ampache plugin didn't work at all with the
format "5.0.0" although that is the format used in the API spec for
versions 5 and 6 and the plugin claims to use APIv6.
This is a difference between the APIv5 `genre_*` functions and the
APIv4 `tag_*` functions. IMHO this is quite pointless, but that's how
the API has been specified.
Added new API methods `bookmarks`, `get_bookmark`, `bookmark_create`,
`bookmark_edit`, `bookmark_delete`. The API spec is currently missing
any examples on these methods so the format of the results was reverse
engineered from the Ampache source code.

As a noteworthy exception, we don't support the optional argument
`date` on the methods `bookmark_create` and `bookmark_edit`. Our
Entity/Mapper/BusinessLayer architecture has been built upon the
principle that the "create" and "update" timestamps get set
automatically on the base class level, and overriding this would
require some design changes which were not seen worthwhile.
The episode already contained both of these information in
human-readable format with keys `filelenght` and `filesize` but the new
fields contain the raw numeric value. This was assumed to be the
purpose of these fields although that is not clearly stated in the API
spec.
Added methods `live_streams` and `live_stream`.
…tream`

The support for the type `playlist` had been added in the API spec
v5.1.0, along with `podcast` and `podcast_episode`.

The types `genre` and `live_stream` should not be supported according to
any existing API spec version but this proprietary extension costs us
nothing.
The APIv5 added a new group "searches" which refers to automatically
generated playlists. The real Ampache server may have multiple such
lists. As per APIv5, the `handshake` response now gives separately the
counts for normal playlists, searches, and for these two combined. The
Kodi Ampache plugin couldn't show the playlists on APIv5 without this
`handshake` change.

Also, the `playlists` method now has the optional argument
`hide_search` which can be used to opt out of seeing the "All tracks"
list. This change came already in the APIv4.3.0.
…array

While using the JSON APIv5+, the responses of "singular" actions (like
`song`, `artist`) are different from "plural" actions (like `songs`,
`artists`). For singular actions, the responses are like:
{ id: "1", name: "Something", ... }
For plural actions (even with just one result), the responses are like:
{ song: [ { id: "1", name: "Something", ... }, ... ] }

With JSON APIv4, the responses of both singular and plural actions are
like:
[ { id: "1", name: "Something", ... }, ... ]

In the XML API, there are no such differences.
…sts`

This has the same limitations as the type `album_artist` in the method
`get_indexes`: When the parameter `album_artist=1` is given, the parameters
`add` and `update` are not supported.
Originally, this function returned results using the `get_indexes`
format but this has been changed in Ampache v6.0.1, see
ampache/ampache#3580.
These actions are now available and support a very limited set of
hard-coded preference values which is just enough for the Ample client.
This is a new action introduced in APIv6, designed as a replacement for
`get_indexes` which is now deprecated.
To get rid of the deprecation warnings from Scrutinizer. The class was
deprecated in NC14 and removed completely in NC26 and that's the whole point
of having this own copy. Our copy is alive and well and not going anywhere in
foreseeable future.
The script hadn't been tested in its latest versions, as most testing
has happened on an old NC version which still uses the obsolete
database.xml.
This is a new action introduced by the APIv6.
It makes sense to utilize the previously introduces BusinessLayer
function `findAllIdsAndNames` also here. This way, we don't need to
construct complete Entity objects just to throw away most of their data.

This makes a clearly measurable difference on the execution time. For
example, the time to list ~2000 songs reduced from around two seconds
to around or under one second.
The change concerns the actions `artists` and `get_indexes` when they
are used to fetch album artists. Now both of these actions support all
the possible arguments also in this case.
The syntax for regular expressions is different on PostgreSQL compared
to MySQL/MariaDB. We now check the DB type from the config (i.e. from
the contents of config.php) and variate the SQL queries accordingly.

Note: Newer versions of SQLite should understand the same regex syntax
as MySQL. But the gotcha is that SQLite is not shipped with a built-in
regex engine and that should be somehow injected as an external
function. That might be impossible for us as the SQLite driver is buried
so deep under the NC/OC abstractions.

Note2: The regex patterns supported on PostgreSQL and MySQL are not
exactly the same although some basic things work the same.
These had got broken when the support for the arg `include` was added
to the actions `artists` and `albums`. Those commits broke the
compatibility of the function interfaces and it hadn't been taken into
account that these functions are called explicitly from `genre_artists`
and `genre_albums`.
The schema of the sessions table has changed.
@paulijar paulijar force-pushed the feature/909_Ampache_API_improvements branch from 1efcaa5 to dc7c238 Compare October 1, 2023 19:34
The SQLite DBMS has such a design flaw that notnull columns without
default value cannot be added to existing tables even when the table in
question is empty. To work around this, the NC table migration logic
now drops the table music_ampache_sessions and recreates it from
scratch.
The placeholder SQL rule `true` used when "exclude childless" not
applicable for the table in question did not work on SQLite. Using
`1=1` instead seems to work.
The method `findAllRated` had no overridden implementation in
AlbumBusinessLayer; hence the "extra fields" were not injected to the
results. Consequently, genre and year were missing from the Ampache
results.
@paulijar paulijar changed the title [WIP] Ampache API improvements Ampache API improvements Oct 5, 2023
@paulijar paulijar merged commit 348a900 into master Oct 5, 2023
@paulijar paulijar deleted the feature/909_Ampache_API_improvements branch October 5, 2023 20:22
@scrutinizer-notifier
Copy link

The inspection completed: 143 updated code elements

@paulijar
Copy link
Collaborator Author

paulijar commented Oct 9, 2023

This is now released as part of Music v1.9.1.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants