Is it possible or just acting stupid ?
You can do it for sure. Whether it is stupid or not is up to you. You will save money on a frame.
See this thread: 20" wheel in a 24" frame... Thoughts?
Summary: It will work but it doesn’t save you much money and switching wheels frequently is a pain.