Introduction
Creating a new CMake project that uses unfamiliar libraries can be daunting and time-consuming. This blog post takes you along on my journey using Copilot to make this easier, and leveraging Custom Instructions to tailor Copilot responses. For a sneak peek, see the outcome of this journey in this repository.
Background
As C++ developers who use Copilot every day, we’ve experienced how it can improve our day-to-day productivity. Copilot really shines in scenarios like new codebases or unfamiliar topics, significantly reducing ramp-up. There are multiple ways to engage with it in Visual Studio, including Completions and Copilot Chat. This blog is focused on leveraging Copilot Chat to be more productive.
What project should I create?
Being a maintainer of the CMake experience, I have always been aware of a popular computer vision library, OpenCV, that is built on CMake to power 2500+ algorithms. However, I’ve never actually played with it and used it in my own code. The first idea that came to mind, being a dog-lover myself, was to create my own real-time dog detector using OpenCV. I set out to create a Dog Detector using CMake and Copilot in Visual Studio.
Import OpenCV into CMake Project
Starting with a new CMake Project in Visual Studio, the first problem to solve was: “How do I import OpenCV into my CMake project?”. Using CMake can sometimes be complex, and I wanted to make sure I used OpenCV in a cross-platform way, so to figure out how to do this, I asked Copilot.
Copilot was able to give me CMakeLists.txt modifications that I can make to my project. However, you’ll notice that it mentions that OpenCV must be installed and that I can use vcpkg. Vcpkg is a cross-platform C/C++ package manager.
Prior to doing this, while the suggested CMakeLists.txt changes are correct, CMake fails the configure step with an error indicating that OpenCV cannot be found. This is because I haven’t yet installed OpenCV.
Use vcpkg to install OpenCV
Now, how do I take Copilot’s suggestion to use vcpkg to install OpenCV? Let’s ask Copilot.
Copilot gave the following response:
You’ll notice that it gave only command-line suggestions for how to install OpenCV with vcpkg. Our team typically uses vcpkg in manifest mode and I’d prefer to use that. Also, you’ll notice that Copilot suggested a CMake command line. We are in Visual Studio, so the command line is driven by our CMake Presets file, and I’d rather make modifications there.
Let’s ask Copilot again with those requests in mind.
Hm, wouldn’t it be nice if I didn’t have to specify manifest mode?
It would also be nice if I didn’t have to specify that I prefer using CMakePresets.json when I ask Copilot questions?
These suggestions are much more aligned with what I’d like to do in my project. Following a combination of the Copilot suggestions and this documentation, I did the following:
- Installed vcpkg and ran the bootstrap script.
- Created the necessary vcpkg.json and vcpkg-configuration.json files defining opencv as a dependency.
- Modified the
CMAKE_TOOLCHAIN_FILEusing the CMakePresets file.
At this point, we were closer, but we still failed to configure. After asking Copilot for help, I realized that it wasn’t finding the toolchain file, and that I should specify the VCPKG_ROOT in an environment variable in the preset and use that variable in the CMAKE_TOOLCHAIN_FILE setting to successfully find it.
At this point, CMake configured successfully and OpenCV had been included in my CMake project.
Use OpenCV to detect dogs
Now that OpenCV was properly included in my CMake project, the next step was to figure out how to use the library and how to get it to detect dogs. From my little background knowledge of OpenCV, I knew that it would need a model trained to detect certain objects. To start, I asked Copilot if OpenCV can detect dogs already, and it told me that there were already publicly available models and, upon further questioning, pointed me to some links where I could download configuration files, weights files, and class names from this repository.
Now that I had downloaded models I could use, I asked Copilot to give me a code snippet that would allow me to detect dogs.
One thing that I hit errors on and had to modify was that Copilot suggested relative paths, and this didn’t work, since it was running the executable from the build location. I found it easier to always use absolute paths to make this simpler.
It would be great if I could tell Copilot to always use absolute paths.
After modifying the paths, I was able to detect dogs in images!
Switch to livestream
I didn’t want to only be able to detect dogs in images, I also wanted to be able to detect dogs using a livestream. I went back to Copilot and asked it to help me make these changes: “How can I modify this code to livestream from a camera rather than reading a single image?”.
Copilot was able to suggest slight modifications to the code, and I was able to integrate a while loop and slightly modify the camera capture mechanism to accomplish this. See the original code versus the suggested livestream code below.
To my surprise, the model could detect other objects as well.
Add UI
To make a more well-rounded application, I wanted to add a user interface to make it easier to control what models are used and to make it easier to operate overall. Through some conversations with Copilot, I was able to add qt5 as a vcpkg.json dependency and add components that allow you to select the .cfg, the .weights, the class names file, and to start and stop the livestream.
The below partial screenshot shows the initial suggestions that copilot gave:
After refining the UI with more conversation with UI and code improvements, the pictures below show The Dog Detector when the camera isn’t on versus when it is.
Also, the ultimate test was to test out the “Dog Detector” with my dog.
Build the project statically
Finally, I wanted to be able to share the executable of this project easily with others so that they could use this project without having to worry about the runtime, the build process, etc. To do this, I asked Copilot: “How to statically link a library with CMake and vcpkg?”
Copilot gave a suggestion to set the vcpkg triplet, but it gave suggestions to modify it in CMakeLists.txt or in the command line. To get better suggestions, I asked again, adding that we want to modify the CMakePresets.json.
This is another instance where it’d be nice to not have to specify this every time I ask.
I took these suggestions and modified them to use CMakePresets.json, so I set the VCPKG_TRIPLET_TARGET in CMakePresets.json.
With these changes, configuration succeeded, but I still got the following build errors:
mismatch detected for ‘RuntimeLibrary’: value ‘MTd_StaticDebug’ doesn’t match value ‘MDd_DynamicDebug’ in DogDetector.cpp.obj.
To help me fix this problem, I asked Copilot to help. Copilot told me to ensure I set the variable OpenCV_STATIC to ON. Since this variable is specific to only the target that includes OpenCV, I set this in the CMakeLists.txt. Additionally, CMake told me that I should set the CMAKE_MSVC_RUNTIME_LIBRARY to MultiThreaded$<$<CONFIG:Debug>:Debug>. However, after making these changes, I was still getting the error.
After much conversation with Copilot, I realized it may have something to do with CMake Policies associated with the CMAKE_MSVC_RUNTIME_LIBRARY. Looking at the documentation, the variable only takes effect when the CMP0091 policy is set to “new”, which is only “new” by default in version 3.15 and beyond.
It would have been great to be informed about CMake Policies that affect CMake variables that Copilot suggested I use.
After updating the CMake project to have a minimum required version of 3.15, the project built statically with no errors.
Custom Instructions
After completing the project, looking back at all the questions and interactions I had with Copilot through Copilot Chat, there were many times where I thought, “Hm, it would be great if I didn’t have to continually specify this information in each question.”. You may have noticed this internal dialog above.
To solve this, I used the Copilot Custom Instructions feature. This feature allows you to create a .github/copilot-instructions.md file that contains additional context you want to be added to every question that you ask Copilot Chat.
Based on the various observations I made, I created a copilot-instructions.md file and added the following custom instructions:
- This project uses vcpkg in manifest mode. Please keep this in mind when giving vcpkg suggestions. Do not provide suggestions like vcpkg install library, as they will not work as expected.
- Prefer setting cache variables and other types of things through CMakePresets.json if possible.
- Give information about any CMake Policies that might affect CMake variables that are suggested or mentioned.
- This project needs to be cross-platform and cross-compiler for MSVC, Clang, and GCC.
- When providing OpenCV samples that use the file system to read files, please always use absolute file paths rather than file names, or relative file paths. For example, use
video.open("C:/project/file.mp4"), notvideo.open("file.mp4")
After adding this file, it was immediately obvious that answers were better when asking the same questions that had been asked previously. For example, I created a new project again and asked how I should use vcpkg to use OpenCV with CMake, and this is part of the response I received.
I was able to repeat the process of creating this project, and by utilizing the Custom Instructions feature, it took me far fewer iterations with Copilot to get answers that I wanted when asking questions related to the custom instructions.
Conclusion
By utilizing the CMake experience along with Copilot in Visual Studio, I was able to quickly get a Dog Detector program up and running, complete with vcpkg integration, an easy-to-use UI, and static linking to share the executable.
I was also able to use the Custom Instructions feature provided by Copilot to make my conversations more useful without having to provide as much repetitive, contextual information in each message to Copilot.
Copilot helped me speed up my development process by answering my questions and helping me navigate CMake and vcpkg. Custom instructions helped Copilot provide more relevant and straightforward answers, without me having to take the time to provide additional context in every question.
To play with the Dog Detector yourself, you can find the source code here, which is released as a sample under MIT license on GitHub.
Disclaimer
The screenshots and responses from Copilot shown above may vary. There is no guarantee that if you attempt to replicate this blog post you will get the same responses.
Author & Team
CMake is a widely adopted cross-platform build system that plays a crucial role in modern C++ development. It is consistently ranked among the top build tools due to its flexibility, scalability, toolability, and extensive ecosystem support. Given its popularity, the team is dedicated to improving its integration, performance, and usability within Microsoft’s products, enabling C++ developers to build, configure, and manage their projects more efficiently. Read more about our CMake support in Visual Studio and VS Code.
Garrett Campbell is a Senior Software Engineer who works on all things CMake, from the Visual Studio CMake experience to the VS Code CMake Tools extension. Also featured in this blog post is Owen. Owen is a mix of lab, Pitbull, and golden retriever!
0 comments
Be the first to start the discussion.