Update DataProvider documentation for beta (#8023)

* Update DataProvider documentation for beta

1 files changed, 112 insertions, 202 deletions

diff --git a/documentation/datamodel/datamodel-providers.asciidoc b/documentation/datamodel/datamodel-providers.asciidoc
index 852c1f0c66..a7acc1e10e 100644
--- a/documentation/datamodel/datamodel-providers.asciidoc
+++ b/documentation/datamodel/datamodel-providers.asciidoc
@@ -14,41 +14,42 @@ A [interfacename]#Listing# is a component that displays one or several propertie
 While each listing component has it's own API for configuring exactly how the data is represented and how it can be manipulated, they all share the same mechanisms for receiving data to show.
 
 The items are generally either loaded directly from memory or lazy loaded from some kind of backend.
-Regardless of how the items are loaded, the component is configured with one or several callbacks or JavaBean property names that define how the item should be displayed.
+Regardless of how the items are loaded, the component is configured with one or several callbacks that define how the item should be displayed.
 
 In the following example, a [classname]#ComboBox# that lists status items is configured to use the [classname]#Status#.[methodname]#getCaption()# method to represent each status.
-There is also a [classname]#Grid#, which is configured with one column from the person's name and another column that converts the year of birth to a string for displaying.
+There is also a [classname]#Grid#, which is configured with one column from the person's name and another showing the year of birth.
 
 [source, java]
 ----
 ComboBox<Status> comboBox = new ComboBox<>();
-comboBox.setItemCaptionProvider(Status::getCaption);
+comboBox.setItemCaptionGenerator(Status::getCaption);
 
 Grid<Person> grid = new Grid<>();
-grid.addColumn("Name", Person::getName);
-grid.addColumn("Year of birth",
-  person -> Integer.toString(person.getYearOfBirth()));
+grid.addColumn(Person::getName).setCaption("Name");
+grid.addColumn(Person::getYearOfBirth)
+  .setCaption("Year of birth");
 ----
 
 [NOTE]
 In this example, it would not even be necessary to define any item caption provider for the combo box if [classname]#Status#.[methodname]#toString()# would be implemented to return a suitable text. [classname]#ComboBox# is by default configured to use [methodname]#toString()# for finding a caption to show.
 
 [NOTE]
-The `Year of birth` column will use [classname]#Grid#'s default [classname]#TextRenderer# which requires the column value to be a [classname]#String#. We could for instance use a [classname]#NumberRenderer# instead, and then the renderer would take care of converting the the number according to its configuration.
+The `Year of birth` column will use [classname]#Grid#'s default [classname]#TextRenderer# which shows any values as a `String`. We could use a [classname]#NumberRenderer# instead, and then the renderer would take care of converting the the number according to its configuration with a formatting setting of our choice.
 
-After we have told the component how the data should be shown, we only need to give it some data to actually show. The easiest way of doing that is as a [interfacename]#java.util.Collection# of item instances.
+After we have told the component how the data should be shown, we only need to give it some data to actually show. The easiest way of doing that is to directly pass the values to show to `setItems`.
 
 [source, java]
 ----
+// Sets items as a collection
 comboBox.setItems(EnumSet.allOf(Status.class));
 
-List<Person> persons = Arrays.asList(
+// Sets items using varargs
+grid.setItems(
   new Person("George Washington", 1732),
   new Person("John Adams", 1735),
   new Person("Thomas Jefferson", 1743),
-  new Person("James Madison", 1751));
-
-grid.setItems(persons);
+  new Person("James Madison", 1751)
+);
 ----
 
 Listing components that allow the user to control the display order of the items are automatically able to sort data by any property as long as the property type implements [classname]#Comparable#.
@@ -59,27 +60,35 @@ We can also define a custom [classname]#Comparator# if we want to customize the
 ----
 grid.addColumn("Name", Person::getName)
   // Override default natural sorting
-  .setValueComparator(
-    Comparator.comparing(String::toLowerCase));
-
-grid.addColumn("Year of birth",
-    person -> Integer.toString(person.getYearOfBirth()))
-  // Sort numerically instead of alphabetically by the string
-  .setItemComparator(
-    Comparator.comparing(Person::getYearOfBirth));
+  .setComparator(Comparator.comparing(
+    person -> person.getName().toLowerCase()));
 ----
 
-With listing components that let the user filter items, we can in the same way define our own [interfacename]#BiPredicate# that is used to decide whether a specific item should be shown when the user has entered a specific text into the text field.
+[NOTE]
+This kind of sorting is only supported for in-memory data.
+Sorting with data that is lazy loaded from a backend is described <<lazy-sorting,later in this chapter>>.
+
+With listing components that let the user filter items, we can in the same way define our own [interfacename]#CaptionFilter# that is used to decide whether a specific item should be shown when the user has entered a specific text into the text field.
+The filter is defined as an additional parameter to `setItems`.
 
 [source, java]
 ----
-comboBox.setFilter((filterText, item) ->
-  item.getCaption().equalsIgnoreCase(filterText));
+comboBox.setItems(
+  (itemCaption, filterText) ->
+    itemCaption.startsWith(filterText),
+  itemsToShow);
 ----
 
+[NOTE]
+This kind of filtering is only supported for in-memory data.
+Filtering with data that is lazy loaded from a backend is described <<lazy-filtering,later in this chapter>>.
+
 Instead of directly assigning the item collection as the items that a component should be using, we can instead create a [classname]#ListDataProvider# that contains the items.
-The list data source can be shared between different components in the same [classname]#VaadinSession# since it is stateless.
-We can also apply different sorting options for each component, without affecting how data is shown in the other components.
+One list data provider instance can be shared between different components to make them show the same data.
+
+We can apply different sorting options for each component using the `sortingBy` method.
+The method creates a new data provider using the same data, but different settings.
+This means that we can apply different sorting options for different components.
 
 [source, java]
 ----
@@ -89,16 +98,16 @@ ListDataProvider<Person> dataProvider =
 ComboBox<Person> comboBox = new ComboBox<>();
 // The combo box shows the person sorted by name
 comboBox.setDataProvider(
-  dataProvider.sortedBy(Person::getName));
+  dataProvider.sortingBy(Person::getName));
 
 Grid<Person> grid = new Grid<>();
 // The grid shows the same persons sorted by year of birth
 grid.setDataProvider(
-  dataProvider.sortedBy(Person::getYearOfBirth));
+  dataProvider.sortingBy(Person::getYearOfBirth));
 ----
 
 The [classname]#Listing# component cannot automatically know about changes to the list of items or to any individual item.
-We must notify the data source when items are changed, added or removed so that components using the data will show the new values.
+We must notify the data provider when items are changed, added or removed so that components using the data will show the new values.
 
 [source, java]
 ----
@@ -107,12 +116,9 @@ ListDataProvider<Person> dataProvider =
 
 Button addPersonButton = new Button("Add person",
   clickEvent -> {
-    // Keep track of the index where the person will be added
-    int addIndex = persons.size();
-
     persons.add(new Person("James Monroe", 1758));
 
-    dataProvider.notifyAdd(addIndex);
+    dataProvider.refreshAll();
 });
 
 Button modifyPersonButton = new Button("Modify person",
@@ -121,13 +127,10 @@ Button modifyPersonButton = new Button("Modify person",
 
     personToChange.setName("Changed person");
 
-    dataProvider.refresh(0);
+    dataProvider.refreshAll();
 });
 ----
 
-[TIP]
-There might be situations where we cannot tell exactly how the data has changed, but only that some parts might have been modified. We can then use the [methodname]#refreshAll()# method, which will make the components reload all the data.
-
 == Lazy Loading Data to a Listing
 
 All the previous examples have shown cases with a limited amount of data that can be loaded as item instances in memory.
@@ -152,7 +155,7 @@ Information about which items to fetch as well as some additional details are ma
 
 [source, java]
 ----
-DataProvider<Person> dataProvider = new BackendDataProvider<>(
+DataProvider<Person, Void> dataProvider = new BackendDataProvider<>(
   // First callback fetches items based on a query
   query -> {
     // The index of the first item to load
@@ -179,6 +182,10 @@ grid.setDataProvider(dataProvider);
 [NOTE]
 The results of the first and second callback must be symmetric so that fetching all available items using the first callback returns the number of items indicated by the second callback. Thus if you impose any restrictions on e.g. a database query in the first callback, you must also add the same restrictions for the second callback.
 
+[NOTE]
+The second type parameter of `DataProvider` defines how the provider can be filtered. In this case the filter type is `Void`, meaning that it doesn't support filtering. Backend filtering will be covered later in this chapter.
+
+[[lazy-sorting]]
 === Sorting
 
 It is not practical to order items based on a [interfacename]#Comparator# when the items are loaded on demand, since it would require all items to be loaded and inspected.
@@ -208,7 +215,7 @@ The sorting options set through the component will be available through [interfa
 
 [source, java]
 ----
-DataProvider<Person> dataProvider = new BackEndDataProvider<>(
+DataProvider<Person, Void> dataProvider = new BackEndDataProvider<>(
   query -> {
     List<PersonSort> sortOrders = new ArrayList<>();
     for(SortOrder<String> queryOrder : query.getSortOrders()) {
@@ -227,7 +234,7 @@ DataProvider<Person> dataProvider = new BackEndDataProvider<>(
       ).stream();
   },
   // The number of persons is the same regardless of ordering
-  query -> persons.getPersonCount()
+  query -> getPersonService().getPersonCount()
 );
 ----
 
@@ -243,12 +250,13 @@ grid.setDataProvider(dataProvider);
 // Will be sortable by the user
 // When sorting by this column, the query will have a SortOrder
 // where getSorted() returns "name"
-grid.addColumn("Name", Person::getName)
+grid.addColumn(Person::getName)
+  .setCaption("Name")
   .setSortProperty("name");
 
 // Will not be sortable since no sorting info is given
-grid.addColumn("Year of birth",
-  person -> Integer.toString(person.getYearOfBirth()));
+grid.addColumn(Person::getYearOfBirth)
+  .setCaption("Year of birth");
 ----
 
 There might also be cases where a single property name is not enough for sorting.
@@ -259,7 +267,7 @@ In such cases, we can define a callback that generates suitable [classname]#Sort
 ----
 grid.addColumn("Name",
     person -> person.getFirstName() + " " + person.getLastName())
-  .setSortBuilder(
+  .setSortOrderProvider(
     // Sort according to last name, then first name
     direction -> Stream.of(
       new SortOrder("lastName", direction),
@@ -267,204 +275,106 @@ grid.addColumn("Name",
     ));
 ----
 
+[[lazy-filtering]]
 === Filtering
 
-A similar approach is also needed with filtering in cases such as [classname]#ComboBox# where the user can control how items are filtered.
-
-The filtering of a data source query is represented as a [interfacename]#BackendFilter# instance. There are existing implementations for some common filtering cases, such as requiring a named property to not be null or a SQL `LIKE` comparison.
-
-[source, java]
-----
-ComboBox<Person> comboBox = new ComboBox<>();
-
-comboBox.setItemCaptionProvider(Person::getName);
-
-comboBox.setFilter(
-  // corresponds to this SQL: WHERE name LIKE [filterText]
-  filterText -> new Like("name", filterText));
-----
-
-If we have a service interface that only supports some specific filtering option, the implementation might become simpler if we define our own [interfacename]#BackendFilter# instead of implementing our backend to use the generic built-in filter types.
+Different types of backends support filtering in different ways.
+Some backends support no filtering at all, some support filtering by a single value of some specific type and some have a complex structure of supported filtering options.
 
-As an example, our service interface with support for filtering could look like this. Ordering support has been omitted in these examples to keep focus on filtering.
+A `DataProvider<Person, String>` supports filtering by string values, but it's up to the implementation to actually define how the filter is actually used.
+It might, for instance, look for all persons with a name beginning with the provided string.
 
-[source, java]
-----
-public interface PersonService {
-  List<Person> fetchPersons(
-    int offset,
-    int limit,
-    String namePrefix);
-  int getPersonCount(String namePrefix);
-}
-----
-
-For the filtering needs of this service, we could define a [classname]#NamePrefixFilter# that corresponds to the only filtering option available.
+You can use the `withFilter` method on a data provider to create a new provider that uses the same data, but applies the given filtering to all queries.
+The original provider instance is not changed.
 
 [source, java]
 ----
-public class NamePrefixFilter implements BackendFilter {
-  private final String prefix;
-
-  public NamePrefixFilter(String prefix) {
-    this.prefix = prefix;
-  }
+DataProvider<Person, String> allPersons = getPersonProvider();
 
-  public String getPrefix() {
-    return prefix;
-  }
-}
-----
+Grid<Person> grid = new Grid<>();
+grid.setDataProvider(allPersons);
 
-In the case of [classname]#ComboBox#, we have to define what kind of [interfacename]#BackendFilter# to use when the user has entered some text that should be used for filtering the displayed items.
+DataProvider<Person, Void> johnPersons = allPersons.withFilter("John");
 
-[source, java]
-----
-comboBox.setFilter(
-  filterText -> new NamePrefixFilter(filterText));
+NativeSelect<Person> johns = new NativeSelect<>();
+johns.setDataProvider(johnPersons);
 ----
+Note that the filter type of the `johnPersons` instance is `Void`, which means that the data provider doesn't support any further filtering.
 
-We can then implement our data source to look for this special filter implementation and pass the name prefix to the service.
-We can create a helper method for handling the filter since the same logic is needed both for fetching and counting items.
+`ListDataProvider` is filtered by callbacks that you can define as lambda expressions, method references or implementations of `SerializablePredicate`.
 
 [source, java]
 ----
-DataProvider<Person> dataProvider = new BackEndDataProvider<>(
-  query -> {
-
-    BackendFilter filter = query.getFilter();
-
-    String namePrefix = filterToNamePrefix(filter);
-
-    return service.fetchPersons(
-        query.getOffset(),
-        query.getLimit(),
-        namePrefix
-      ).stream();
-  },
-  query -> persons.getPersonCount(
-    filterToNamePrefix(query.getFilter))
-);
-
-public static String filterToNamePrefix(BackendFilter filter) {
-  if (filter == null) {
-    return null;
-  }
+ListDataProvider<Person> allPersons =
+  new ListDataProvider<>(persons);
 
-  if (filter instanceof NamePrefixFilter)) {
-    return ((NamePrefixFilter) filter).getPrefix();
-  } else {
-    throw new UnsupportedOperationException(
-      "This data source only supports NamePrefixFilter");
-  }
-}
+Grid<Person> grid = new Grid<>();
+grid.setDataProvider(allPersons.withFilter(
+  person -> person.getName().startsWith("John")
+));
 ----
-
 [TIP]
-If the amount of data in the backend is small enough, it might be better to load all the items into a list and use a [classname]#ListDataProvider# instead of implementing filtering or sorting support in a custom [classname]#DataProvider# class and configuring the components accordingly.
+`ListDataProvider` lets you combine multiple filters since the return value of `withFilter` is itself also filterable by `SerializablePredicate`.
+
+A listing component that lets the user control how the displayed data is filtered has some specific filter type that it uses.
+For `ComboBox`, the filter is the `String` that the user has typed into the search field.
+This means that `ComboBox` can only be used with a data provider whose filtering type is `String`.
 
-We can also create a base data source and then use different variations for different components, similarly to the previous examples with [classname]#ListDataProvider#.
+To use a data provider that filters by some other type, you need to use the `convertFilter`.
+This method creates a new data provider that uses the same data but a different filter type; converting the filter value before passing it to the original data provider instance.
 
 [source, java]
 ----
-DataProvider<Person> dataProvider = ...
+DataProvider<Person, String> allPersons = getPersonProvider();
+ListDataProvider<Person> listProvider = new ListDataProvider<>(persons);
 
-grid.setDataProvider(dataProvider
-  .filteredBy(new Like("name", "Ge%"))
-  .sortedBy(new SortOrder(
-    "yearOfBirth", SortDirection.ASCENDING)));
+ComboBox<Person> comboBox = new ComboBox();
 
-comboBox.setDataProvider(dataProvider
-  .sortedBy(new SortOrder(
-    "name", SortOrder.DESCENDING)));
+// Can use DataProvider<Person, String> directly
+comboBox.setDataProvider(allPersons);
 
+// Must define how to convert from a string to a predicate
+comboBox.setDataProvider(listProvider.convertFilter(
+  filterText -> {
+    // Create a predicate that filters persons by the given text
+    return person -> person.getName().contains(filterText);
+  }
+));
 ----
 
-=== Special Fetching Cases
-
-In some cases it might be necessary directly extend [classname]#BackendDataProvider# instead of constructing an instance based the two simple callback methods shown above.
+To create a data provider that supports filtering, you only need to look for a filter in the provided query and use that filter when fetching and counting items. `withFilter` and `convertFilter` are automatically implemented for you.
 
-One such case is if the backend loads items based on a page index and a page size so that the start index in the query always needs to be a multiple of the page size. As an example, our service interface made for paging could look like this.
+As an example, our service interface with support for filtering could look like this. Ordering support has been omitted in this example to keep focus on filtering.
 
 [source, java]
 ----
 public interface PersonService {
   List<Person> fetchPersons(
-    int pageIndex,
-    int pageSize);
-  int getPersonCount();
-}
-----
-
-We can use this kind of backend service as long as we also make the data source declare that queries should always be done for whole pages.
-Components using this data source will take the information into account when querying for data.
-
-[source, java]
-----
-public class PersonDataProvider
-        extends BackendDataProvider<Person> {
-
-  @Override
-  public boolean alignQueries() {
-    // Informs the part that fetches items that the query offset
-    // must be a multiple of the query limit, i.e. that only full
-    // pages should be requested
-    return true;
-  }
-
-  @Override
-  public void fetch(Query<Person> query,
-          FetchResult<Person> result) {
-    int pageSize = query.getLimit();
-
-    // Caller guarantees that query.getOffset() % pageSize == 0
-    int pageIndex = query.getOffset() / pageSize;
-
-    result.setItems(getPersonService().fetchPersons(pageIndex, pageSize));
-  }
-
-  @Override
-  public int getCount(Query<Person> query) {
-    return getPersonService().getPersonCount();
-  }
+    int offset,
+    int limit,
+    String namePrefix);
+  int getPersonCount(String namePrefix);
 }
 ----
 
-Some backends may also have limitations on how many (or few) items can be fetched at once.
-While our data source implementation could deal with that limitation internally by sending multiple requests to the backend and then assembling the results together before returning the result, we can also make the data source indicate that the responsibility for splitting up the query is on the caller instead.
+A data provider using this service could use `String` as its filtering type.
+It would then look for a string to filter by in the query and pass it to the service method.
 
 [source, java]
 ----
-public class PersonDataProvider
-        extends BackendDataProvider<Person> {
-
-  @Override
-  public int getMaxLimit() {
-    // Informs the part that fetches items that the maximum
-    // supported query limit size is 30
-    return 30;
-  }
-
-  @Override
-  public void fetch(Query<Person> query,
-          FetchResult<Person> result) {
-    List<Person> persons = getPersonService().fetchPersons(
+DataProvider<Person, String> dataProvider = new BackEndDataProvider<>(
+  query -> {
+    // getFilter returns Optional<String>
+    String filter = query.getFilter().orElse(null);
+    return getPersonService().fetchPersons(
       query.getOffset(),
-      query.getLimit());
-    result.setItems(persons);
-  }
-
-  @Override
-  public int getCount(Query<Person> query) {
-    return getPersonService().getPersonCount();
+      query.getLimit(),
+      filter
+    ).stream();
+  },
+  query -> {
+    String filter = query.getFilter().orElse(null);
+    return getPersonService().getPersonCount(filter);
   }
-}
+);
 ----
-
-[TIP]
-You can set the max limit and the min limit to the same value if you are using a backend that has a hardcoded page size. You can also combine this with aligned queries.
-
-Yet another case that benefits from custom querying options is backends that perform better if items are fetched relative to a previously executed query instead of by skipping items based on an absolute offset.
-
-To help with this, the provided query object will automatically contain a reference to the item immediately before the start of the first new item to fetch if available.
-The item immediately after the end of the range to fetch might also be available in some cases if the user is scrolling through the data backwards. There are, however, no guarantees that either item will be available in all queries, so the implementation should always also support fetching by offset.