What I am looking for is a way to: move left pane to right, move bottom right pane to top left corner etc.

The method I use to move windows around is similar to what I do with text -- "d" the window, then "p" it elsewhere. Except the mappings are <c-w>d and <c-w>p: https://github.com/andrewradev/yankwin.vim

I doubt this plugin would be convenient to you out of the box, since for instance it doesn't paste vertical splits, and if you have lots of windows, maybe you use a large monitor with them. I'd say you could use it as an idea at least. The core of it is as simple as this: https://github.com/AndrewRadev/vim-lectures/blob/4bb97b3a2cb27fb8542d494079ee2ffde4a49d93/snippets/03-winmove.vim

If you wanted to use the mouse, you could map <LeftMouse>, <MiddleMouse>, and <RightMouse> for it. Possibly with ctrl or some other modifier. There's even <LeftDrag>, but I haven't experimented with it, so I can't say if it would be easy to implement drag & drop for windows. :help mouse-overview.