Pulling Maps From Sequences Requires Named Keys

Apr 21, 2025 by ADMIN 48 views

Problem

When working with sequences of maps in Clojure, it can be challenging to pull specific data from these sequences. The lasagna-pull function requires all map keys to be specified up front, which can be limiting when working with dynamic or unknown schemas. This issue is particularly relevant when working with data storage systems like Datomic or re-frame, where data may not follow fixed schemas.

Impact

The current implementation of lasagna-pull makes it difficult to work with lists of entities that may not follow fixed schemas. This limitation can lead to cumbersome workarounds and decreased productivity. For instance, when working with Datomic or re-frame, it is common to encounter data that does not conform to a fixed schema. In such cases, the inability to pull specific data from a sequence of maps can be a significant hindrance.

Probable Cause

The root cause of this issue lies in the way schema/pattern-schema-of handles [:sequential :map] schemas. When a schema is defined as a vector of maps, the resulting pattern schema does not accurately reflect the structure of the data. This is evident in the following example:

(def data-schema
  [:vector :map])

(def pattern-schema
  (schema/pattern-schema-of data-schema))

pattern-schema
;; => [:cat {:sg.flybot.pullable.schema/pattern? true}
;; =>  :map
;; =>  [:? [:fn #function[lvar?]]]
;; =>  [:? [:alt [:cat [:= :seq] [:vector {:min 1, :max 2} :int]]]]]

(malli.core/validate pattern-schema ['?]) ;=> false
(malli.core/validate pattern-schema [{} '?]) ;=> true

As shown in the example, the resulting pattern schema does not accurately reflect the structure of the data. This is because schema/pattern-schema-of does not handle [:sequential :map] schemas correctly.

Suggestion

To address this issue, it would be beneficial to allow using ['?]' to pull whole elements (such as entire maps) from a sequence. This would enable developers to work with sequences of maps in a more flexible and efficient manner. For example:

((pull/query ['?]) [{:a 1} {:a 2 :b 1}])
;; => [{:a 1} {:a 2 :b 1}]

By making this change, developers would be able to pull specific data from sequences of maps without having to specify all map keys up front. This would greatly simplify the process of working with dynamic or unknown schemas, making it easier to develop robust and efficient applications.

Benefits

Allowing the use of ['?]' to pull whole elements from a sequence would have several benefits. Firstly, it would enable developers to work with sequences of maps in a more flexible and efficient manner. This would be particularly useful when working with data storage systems like Datomic or re-frame, where data may not follow fixed schemas.

Secondly, this change would simplify the process of working with dynamic or unknown schemas. By allowing developers to pull specific data from a sequence without having to specify all map keys up front, it would be easier to develop robust and efficient applications.

Implementation

To implement this change, the lasagna-pull function would need to be modified to allow the use of ['?]' to pull whole elements from a sequence. This would involve updating the function to recognize the ['?]' pattern and return the entire map element.

Here is an example of how the modified lasagna-pull function might look:

(defn lasagna-pull
  [pattern data]
  (cond
    (and (sequential? pattern) (map? (first pattern)))
    (map (fn [x] (lasagna-pull (first pattern) x)) data)
    (and (sequential? pattern) (not (map? (first pattern))))
    (map (fn [x] (lasagna-pull (first pattern) x)) data)
    :else
    (lasagna-pull (first pattern) data)))

This modified function would allow the use of ['?]' to pull whole elements from a sequence, making it easier to work with sequences of maps in a flexible and efficient manner.

Conclusion

Q: What is the current issue with pulling maps from sequences in Clojure?

A: The current implementation of lasagna-pull requires all map keys to be specified up front, which can be limiting when working with dynamic or unknown schemas.

Q: Why is this issue relevant to developers working with Datomic or re-frame?

A: When working with Datomic or re-frame, data may not follow fixed schemas, making it difficult to work with lists of entities that may not follow fixed schemas.

Q: What is the probable cause of this issue?

A: The root cause of this issue lies in the way schema/pattern-schema-of handles [:sequential :map] schemas.

Q: How does the current implementation of `schema/pattern-schema-of` handle `[:sequential :map]` schemas?

A: The current implementation of schema/pattern-schema-of does not accurately reflect the structure of the data when handling [:sequential :map] schemas.

Q: What is the suggested solution to this issue?

A: To address this issue, it would be beneficial to allow using ['?]' to pull whole elements (such as entire maps) from a sequence.

Q: How would this change simplify the process of working with dynamic or unknown schemas?

A: By allowing developers to pull specific data from a sequence without having to specify all map keys up front, it would be easier to develop robust and efficient applications.

Q: What are the benefits of allowing the use of `['?]'` to pull whole elements from a sequence?

A: The benefits of allowing the use of ['?]' to pull whole elements from a sequence include:

Enabling developers to work with sequences of maps in a more flexible and efficient manner
Simplifying the process of working with dynamic or unknown schemas
Making it easier to develop robust and efficient applications

Q: How would the `lasagna-pull` function need to be modified to allow the use of `['?]'` to pull whole elements from a sequence?

A: The lasagna-pull function would need to be modified to recognize the ['?]' pattern and return the entire map element.

Q: What is an example of how the modified `lasagna-pull` function might look?

A: Here is an example of how the modified lasagna-pull function might look:

(defn lasagna-pull
  [pattern data]
  (cond
    (and (sequential? pattern) (map? (first pattern)))
    (map (fn [x] (lasagna-pull (first pattern) x)) data)
    (and (sequential? pattern) (not (map? (first pattern))))
    (map (fn [x] (lasagna-pull (first pattern) x)) data)
    :else
    (lasagna-pull (first pattern) data)))

Q: What is the conclusion of this article?

A: In conclusion, the current implementation of lasagna-pull requires all map keys to be specified up front, which can be limiting when working with dynamic or unknown schemas. To address this issue, it would be beneficial to allow using ['?]' to pull whole elements from a sequence. This would enable developers to work with sequences of maps in a more flexible and efficient manner, making it easier to develop robust and efficient applications.