Allow Overriding The Copy-on-write Behaviou Of Adapters

Introduction

In the current implementation of adapters in Psycopg, objects created inherit the adapters configuration from their parent objects. This operation is implemented via a shallow copy, ensuring that the children remain unchanged even if the parent's adapters map is modified. While this default behaviour follows the principle of least surprise, it can sometimes hinder the flexibility of the system. In this article, we will explore the possibility of adding a propagate method to the register_dumper() and register_loader() methods, allowing users to override the copy-on-write behaviour of adapters.

Current Behaviour

The current implementation of adapters in Psycopg is designed to ensure that the children remain unchanged even if the parent's adapters map is modified. This is achieved through a shallow copy of the adapters map. When a parent's adapters map is changed, a copy is made to keep its children unchanged. This behaviour is documented in the Psycopg documentation, specifically in the advanced adapt section.

Limitations of the Current Behaviour

While the current behaviour follows the principle of least surprise, it can sometimes get in the way of the system's flexibility. For example, when using a connection pool, registering an enum requires creating an auxiliary connection before creating the pool, registering the enum globally, and then creating the pool. This can be cumbersome and may lead to confusion. Another case where a propagation behaviour would be useful is when updating adapters after a query is executed, so that people could change the cursor's adapter without the need to expose the transformer object.

Proposal: Adding a propagate Method

To address the limitations of the current behaviour, I propose adding a propagate method to the register_dumper() and register_loader() methods, as well as related more specialised functions such as register_hstore() and register_enum(). The default value of this method would be False, maintaining the current behaviour. However, by setting it to True, the copy step would be avoided, and the current map would be updated inplace, changing the map for all objects already created.

Benefits of the Proposal

The proposed addition of a propagate method would provide several benefits:

• Flexibility: Users would have the option to override the copy-on-write behaviour of adapters, allowing for more flexibility in their use of Psycopg.
• Simplified Usage: The need to create auxiliary connections and register enums globally would be eliminated, simplifying the usage of connection pools.
• Improved Performance: By avoiding the copy step, the performance of the system would be improved, especially in scenarios where adapters are frequently updated.

Implementation

To implement the proposed propagate method, the following changes would be required:

• Add a propagate parameter to the register_dumper() and register_loader() methods, as well as related more specialised functions.
• Update the implementation of these methods to check the value of the propagate parameter. If it is True, the current map would be updated inplace, avoiding the copy step.
• Document the new propagate method and its usage in the Psycopg documentation.

Conclusion

In conclusion, the proposed addition of a propagate method to the register_dumper() and register_loader() methods would provide several benefits, including flexibility, simplified usage, and improved performance. While the current behaviour follows the principle of least surprise, it can sometimes hinder the flexibility of the system. By allowing users to override the copy-on-write behaviour of adapters, Psycopg would become more user-friendly and efficient.

Future Work

Future work could involve:

• Testing: Thorough testing of the new propagate method to ensure its correctness and performance.
• Documentation: Updating the Psycopg documentation to reflect the new propagate method and its usage.
• Community Feedback: Seeking feedback from the Psycopg community to ensure that the proposed change meets their needs and expectations.

Introduction

In our previous article, we explored the possibility of adding a propagate method to the register_dumper() and register_loader() methods in Psycopg, allowing users to override the copy-on-write behaviour of adapters. In this article, we will answer some frequently asked questions (FAQs) about this proposal.

Q: What is the current copy-on-write behaviour of adapters in Psycopg?

A: The current implementation of adapters in Psycopg is designed to ensure that the children remain unchanged even if the parent's adapters map is modified. This is achieved through a shallow copy of the adapters map. When a parent's adapters map is changed, a copy is made to keep its children unchanged.

Q: Why is the current copy-on-write behaviour necessary?

A: The current behaviour follows the principle of least surprise, which means that it behaves in a way that is expected by most users. This principle is important in software development, as it helps to ensure that users can easily understand and use a system.

Q: What are the limitations of the current copy-on-write behaviour?

A: While the current behaviour follows the principle of least surprise, it can sometimes get in the way of the system's flexibility. For example, when using a connection pool, registering an enum requires creating an auxiliary connection before creating the pool, registering the enum globally, and then creating the pool. This can be cumbersome and may lead to confusion.

Q: What is the proposed propagate method, and how does it work?

A: The proposed propagate method is a new parameter that would be added to the register_dumper() and register_loader() methods, as well as related more specialised functions. If set to True, the current map would be updated inplace, avoiding the copy step. This would allow users to override the copy-on-write behaviour of adapters.

Q: What are the benefits of the proposed propagate method?

A: The proposed propagate method would provide several benefits, including:

• Flexibility: Users would have the option to override the copy-on-write behaviour of adapters, allowing for more flexibility in their use of Psycopg.
• Simplified Usage: The need to create auxiliary connections and register enums globally would be eliminated, simplifying the usage of connection pools.
• Improved Performance: By avoiding the copy step, the performance of the system would be improved, especially in scenarios where adapters are frequently updated.

Q: How would the proposed propagate method be implemented?

A: To implement the proposed propagate method, the following changes would be required:

• Add a propagate parameter to the register_dumper() and register_loader() methods, as well as related more specialised functions.
• Update the implementation of these methods to check the value of the propagate parameter. If it is True, the current map would be updated inplace, avoiding the copy step.
• Document the new propagate method and its usage in the Psycopg documentation.

: What are the next steps for implementing the proposed propagate method?

A: The next steps for implementing the proposed propagate method would involve:

• Testing: Thorough testing of the new propagate method to ensure its correctness and performance.
• Documentation: Updating the Psycopg documentation to reflect the new propagate method and its usage.
• Community Feedback: Seeking feedback from the Psycopg community to ensure that the proposed change meets their needs and expectations.

Conclusion

In conclusion, the proposed propagate method would provide several benefits, including flexibility, simplified usage, and improved performance. By allowing users to override the copy-on-write behaviour of adapters, Psycopg would become a more user-friendly and efficient system. We hope that this Q&A article has provided a clear understanding of the proposal and its benefits.