That thing is just a "core plug".
This is a characteristic of every engine is constructed using sand-forms to create the internal passages necessary for coolant and oil flow and other items.
Every sand-cast engine block or head has one or more of these holes that are included in the design to allow quick, easy and complete removal of the sand-form after the casting process is finished.
So yeah, just use something like JBweld to stick it back in there for good.
https://en.wikipedia.org/wiki/Core_plug