Build Gluten On Mac Issue

by ADMIN 26 views

=====================================

Problem Description

When trying to build Gluten on a Mac, you may encounter several issues. In this article, we will discuss two common problems and their solutions.

1. CMake Minimum Required Issue

The first issue is related to the CMake minimum required version. You may encounter the following error message:

CMake Error at _build/_deps/googletest-src/CMakeLists.txt:4 (cmake_minimum_required): Compatibility with CMake < 3.5 has been removed from CMake.

Update the VERSION argument <min> value. Or, use the <min>...<max> syntax to tell CMake that the project requires at least <min> but has been updated to work with policies introduced by <max> or earlier.</max></min></max></min></min>

Or, add -DCMAKE_POLICY_VERSION_MINIMUM=3.5 to try configuring anyway.

This error occurs because the CMake version used in the project is older than the minimum required version. To solve this issue, you need to update the CMake version or add the -DCMAKE_POLICY_VERSION_MINIMUM=3.5 flag when configuring the project.

2. -Wno-deprecated-declarations -Wno-inconsistent-missing-override -Wno-macro-redefined -UUNLIKELY Issue

The second issue is related to the deprecated declarations and inconsistent missing override warnings. You may encounter the following error message:

In file included from /usr/local/include/thrift/TApplicationException.h:23:
/usr/local/include/thrift/Thrift.h:54:19: error: 'iterator<std::forward_iterator_tag, std::pair<int, const char *>>' is deprecated [-Werror,-Wdeprecated-declarations]
   54 |     : public std::iterator<std::forward_iterator_tag, std::pair<int, const char*> > {
      |                   ^
/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk/usr/include/c++/v1/__iterator/iterator.h:23:29: note: 'iterator<std::forward_iterator_tag, std::pair<int, const char *>>' has been explicitly marked deprecated here
   23 | struct _LIBCPP_TEMPLATE_VIS _LIBCPP_DEPRECATED_IN_CXX17 iterator {
      |                             ^
/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk/usr/include/c++/v1/__config:1022:41: note: expanded from macro '_LIBCPP_DEPRECATED_IN_CXX17'
 1022 | #    define _LIBCPP_DEPRECATED_IN_CXX17 _LIBCPP_DEPRECATED
      |                                         ^
/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk/usr/include/c++/v1/__config:995:49: note: expanded from macro '_LIBCPP_DEPRECATED'
  995 | #      define _LIBCPP_DEPRECATED __attribute__((__deprecated__))
      |                                                 ^
In file included from /Users/yumwang/work/gluten-bhl/ep/build-velox/build/velox_ep/velox/dwio/parquet/reader/PageReader.cpp:27:
/usr/local/include/thrift/protocol/TCompactProtocol.h:143:7: error: 'getMinSerializedSize' overrides a member function but is not marked 'override' [-Werror,-Winconsistent-missing-override]
  143 |   int getMinSerializedSize(TType type);
      |       ^
/usr/local/include/thrift/protocol/TCompactProtocol.h:266:18: note: in instantiation of template class 'apache::thrift::protocol::TCompactProtocolT<apache::thrift::transport::TTransport>' requested here
  266 |       prot = new TCompactProtocol(trans, string_limit_, container_limit_);
      |                  ^
/usr/local/include/thrift/protocol/TProtocol.h:547:15: note: overridden virtual function is here
  547 |   virtual int getMinSerializedSize(TType type) {
      |               ^
In file included from /Users/yumwang/work/gluten-bhl/ep/build-velox/build/velox_ep/velox/dwio/parquet/reader/PageReader.cpp:27:
/usr/local/include/thrift/protocol/TCompactProtocol.h:145:8: error: 'checkReadBytesAvailable' overrides a member function but is not marked 'override' [-Werror,-Winconsistent-missing-override]
  145 |   void checkReadBytesAvailable(TSet& set)
      |        ^
/usr/local/include/thrift/protocol/TProtocol.h:558:16: note: overridden virtual function is here
  558 |   virtual void checkReadBytesAvailable(TSet& set)
      |                ^
In file included from /Users/yumwang/work/gluten-bhl/ep/build-velox/build/velox_ep/velox/dwio/parquet/reader/PageReader.cpp:27:
/usr/local/include/thrift/protocol/TCompactProtocol.h:150:8: error: 'checkReadBytesAvailable' overrides a member function but is not marked 'override' [-Werror,-Winconsistent-missing-override]
  150 |   void checkReadBytesAvailable(TList& list)
      |        ^
/usr/local/include/thrift/protocol/TProtocol.h:563:16: note: overridden virtual function is here
  563 |   virtual void checkReadBytesAvailable(TList& list)
      |                ^
In file included from /Users/yumwang/work/gluten-bhl/ep/build-velox/build/velox_ep/velox/dwio/parquet/reader/PageReader.cpp:27:
/usr/local/include/thrift/protocol/TCompactProtocol.h:155:8: error: 'checkReadBytesAvailable' overrides a member function but is not marked 'override' [-Werror,-Winconsistent-missing-override]
  155 |   void checkReadBytesAvailable(TMap& map)
      |        ^
/usr/local/include/thrift/protocol/TProtocol.h:568:16: note: overridden virtual function is here
  568 |   virtual void checkReadBytesAvailable(TMap& map)
      |                ^
9 errors generated.

This error occurs because the Thrift library is using deprecated declarations and inconsistent missing override warnings. To solve this issue, you need to update the Thrift library or add the necessary flags to suppress the warnings.

System Information

The system information is as follows:

 % dev/info.sh    
dev/info.sh: line 47: lscpu: command not found

Gluten Version: 1.5.0-SNAPSHOT
Commit: 88dbb0eb8837aad9d380269bc37d9f0fb26b57b0
CMake Version: 4.0.1
System: Darwin-24.3.0
Arch: arm64
CPU Name: 
C++ Compiler: /usr/bin/c++
C++ Compiler Version: 16.0.0.16000026
C Compiler: /usr/bin/cc
C Compiler Version: 16.0.0.16000026
CMake Prefix Path: /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk/usr;/opt/homebrew;/usr/local;/usr;/;/opt/homebrew;/usr/local;/usr/X11R6;/usr/pkg;/opt;/sw;/opt/local

CMake Log

The CMake log is as follows:

Note: The CMake log is empty because the CMake configuration was not successful.

Solution

To solve the issues mentioned above, you need to update the CMake version and the Thrift library. You also need to add the necessary flags to suppress the warnings.

Update CMake Version

To update the CMake version, you can use the following command:

brew install cmake

This will install the latest version of CMake.

Update Thrift Library

To update the Thrift library, you can use the following command:

brew install thrift

This will install the latest version of the Thrift library.

Add Flags to Suppress Warnings

To add the flags to suppress the warnings, you can use the following command:

cmake -DCMAKE_POLICY_VERSION_MINIMUM=3.5 -DCMAKE_CXX_FLAGS="-Wno-deprecated-declarations -Wno-inconsistent-missing-override -Wno-macro-redefined -UUNLIKELY" .

This will add the necessary flags to suppress the warnings.

Conclusion

In this article, we discussed two common issues that may occur when building Gluten on a Mac. We also provided the solutions to these issues. By updating the CMake version and the Thrift library, and adding the necessary flags to suppress the warnings, you can successfully build Gluten on your Mac.

Future Work

In the future, we plan to update the Gluten documentation to include the necessary information to build Gluten on a Mac. We also plan to provide more detailed instructions on how to update the CMake version and the Thrift library.

References

=============================================

Q: What is Gluten?

A: Gluten is a C++ library for building high-performance data processing systems. It provides a set of tools and APIs for building data processing pipelines, including data ingestion, processing, and output.

Q: What are the system requirements for building Gluten on Mac?

A: The system requirements for building Gluten on Mac are:

  • macOS 10.14 or later
  • CMake 3.5 or later
  • Thrift 0.14 or later
  • C++ compiler (e.g. clang or gcc)

Q: How do I install CMake on Mac?

A: You can install CMake on Mac using Homebrew by running the following command:

brew install cmake

Q: How do I install Thrift on Mac?

A: You can install Thrift on Mac using Homebrew by running the following command:

brew install thrift

Q: What are the common issues that may occur when building Gluten on Mac?

A: The common issues that may occur when building Gluten on Mac include:

  • CMake minimum required version issue
  • Deprecated declarations and inconsistent missing override warnings
  • Missing dependencies

Q: How do I resolve the CMake minimum required version issue?

A: To resolve the CMake minimum required version issue, you can update the CMake version by running the following command:

brew install cmake

You can also add the -DCMAKE_POLICY_VERSION_MINIMUM=3.5 flag when configuring the project.

Q: How do I resolve the deprecated declarations and inconsistent missing override warnings?

A: To resolve the deprecated declarations and inconsistent missing override warnings, you can update the Thrift library by running the following command:

brew install thrift

You can also add the -DCMAKE_CXX_FLAGS="-Wno-deprecated-declarations -Wno-inconsistent-missing-override -Wno-macro-redefined -UUNLIKELY" flag when configuring the project.

Q: How do I resolve the missing dependencies issue?

A: To resolve the missing dependencies issue, you can check the project's dependencies and install the missing ones. You can also use the brew install command to install the missing dependencies.

Q: How do I build Gluten on Mac?

A: To build Gluten on Mac, you can follow these steps:

  1. Install CMake and Thrift using Homebrew.
  2. Clone the Gluten repository using Git.
  3. Configure the project using CMake.
  4. Build the project using the make command.
  5. Install the project using the make install command.

Q: How do I run Gluten on Mac?

A: To run Gluten on Mac, you can follow these steps:

  1. Build the project using the make command.
  2. Run the project using the ./gluten command.

Note: The ./gluten command will run the Gluten server, which will start listening on port 8080.

Q: How do I troubleshoot Gluten on Mac?

A: To troubleshoot Gluten on Mac, you can follow these steps:

  1. Check the project's logs for any errors or warnings.
  2. Check the system logs for any errors or warnings related to Gluten.
  3. Use the brew doctor command to check the system for any issues related to Homebrew.
  4. Use the cmake --debug command to debug the CMake configuration.

Q: How do I get help with Gluten on Mac?

A: To get help with Gluten on Mac, you can follow these steps:

  1. Check the Gluten documentation for any information related to building and running Gluten on Mac.
  2. Check the Gluten community forums for any discussions related to building and running Gluten on Mac.
  3. Contact the Gluten developers for any questions or issues related to building and running Gluten on Mac.

Note: The Gluten community forums and the Gluten developers' contact information can be found on the Gluten website.